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1 .  Introduction 


The  Tactical  Nephanalysis  (TACNEPH)  program  was  a  four-year  research  effort 
conducted  by  Atmospheric  and  Environmental  Research,  Inc.  (AER)  for  the  Satellite 
Meteorology  Branch  of  the  Philhps  Laboratory  Geophysics  Directorate  (PL/GPAS)  and 
sponsored  by  the  Defense  Meteorological  Satellite  Program  (DMSP)  Systems  Program 
Office  (SPO),  Space  and  Missile  Systems  Center  (SMC),  Los  Angeles,  CA.  The 
TACNEPH  objective  was  to  develop  and  validate  a  set  of  robust,  relocatable  regional  cloud 
(neph)  detection  and  analysis  algorithms  capable  of  generating  gridded  fields  of  analyzed 
cloud  amount  and  altitude  using  only  the  resources  available  on  a  tactical  terminal  such  as 
Mark  IV-B  or  the  Small  Tacticd  Terminal  (STT).  Key  requirements  are  global  applica¬ 
bility,  the  ability  to  assimilate  data  from  bodi  military  and  civilian  polar  orbiting  satellites  in 
real-time,  and  support  for  a  range  of  available  sensor  data  required  to  accommodate  data- 
impaired  or  data-denied  situations. 

Required  attributes  of  the  TACNEPH  analysis  are  autonomy,  transportability,  relia¬ 
bility,  and  graceful  degradation.  Autonomy  requires  that  the  cloud  analysis  be  performed 
without  timely  updates  to  supporting  data  from  a  central  site  such  as  the  Air  Force  Global 
Weather  Central  (AFGWC).  Transportability  means  that  the  TACNEPH  cloud  algorithms 
must  operate  within  the  constraints  of  tactical  terminal  ingest,  computing,  and  display 
capabilities.  Reliability  demands  that  the  algorithms  operate  successfully  for  any  location 
around  the  world  over  a  range  of  data  availability  conditions.  Graceful  degradation  implies 
that  TACNEPH  algorithms  adapt  to  changes  in  available  satellite  data  resources  to  produce 
the  most  accurate  analysis  possible  under  all  conditions.  The  TACNEPH  heritage  he.s  in 
the  global  RTNEPH  model  which,  along  with  its  predecessor  the  3DNEPH,  the  Air  Force 
has  operated  continuously  for  over  20  years  (Hamill  et  al.,  1992).  However,  while  the 
fundamental  requirement  to  operationally  analyze  satellite  sensor  data  to  obtain  cloud 
information  is  the  same  for  both  models,  TACNEPH  requirements  depart  from  RTNEPH 
capabilities  in  a  number  of  areas.  Important  differences  are  the  regional  vs.  global  nature  of 
the  models,  the  TACNEPH  requirements  to  exploit  multiple  sensor  data  sources  and  to 
operate  in  &e  absence  of  supporting  databases  from  non-satellite  sources,  and  the 
processing  environments  in  which  die  two  models  operate. 

Relevant  dynamic  data  resources  assumed  to  be  available  to  a  tactical  terminal 
include  digital  satellite  sensor  data  from:  DMSP  Operational  Linescan  System  (OLS), 
Special  Sensor  Microwave  Imager  and  Temperature  Sounder  (SSM/I  and  SSM/T, 
respectively),  and  National  Oceanic  and  Atmospheric  Administration  (NOAA)  Advanced 
Very  High  Resolution  Radiometer  (AVHRR).  Static  climatological  databases  of  surface 
shelter  temperature  and  1000  mb  geopotentii  heights  along  with  a  rudimentary  geographic 
type  database  are  also  assumed  to  be  carried  on  the  tactical  terminal.  Conventional  surface 
and  upper  air  observations  are  assumed  to  be  unavailable  on  the  tactical  terminal,  but  may 
be  available  at  a  nearby  or  collocated  facility  (e.g.  Combat  Weather  System,  CWS).  Other 
potentially  useful  satellite  data  sources  are  not  currently  used  by  TACNEPH;  these  include 
the  NOAA  TIROS  Operational  Vertical  Sounder  (TOVS)  package  and  all  geostationary 
environmental  satellites  (both  U.S.  and  foreign). 

Required  TACNEPH  cloud  parameters  are  cloud  amount  and  altitude. 

Experimental  algorithms  to  retrieve  cloud  thickness  and  base  from  OLS  and  AVHRR  data 
were  also  developed.  Cloud  attributes  are  reported  on  a  pixel  level  and,  as  such,  can  be 
adapted  to  any  gridded  field  format  that  may  be  required  by  a  particular  application. 
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1.1  TACNEPH  Program  Task  Requirements 

During  the  TACNEPH  research  and  development  program  work  was  performed  on 
new  algorithm  development,  validation,  testing  and  software  maintenance.  Nine  tasks, 
some  with  multiple  subtasks,  were  originally  identified  as  necessary  to  successfully  com¬ 
plete  the  program  requirements.  Subsequent  to  the  start  of  the  program  one  task  involving 
evaluation  of  a  government  furnished  SSM/1  cloud  analysis  algorithm  was  dropped  when 
results  from  other  investigations  demonstrated  that  the  algorithm  was  unsuitable  for 
TACNEPH  applications.  Midway  through  the  development  effort  it  was  recognized  that 
insufficient  satellite  data  were  av^able  to  the  program  for  testing  and  validation  of  the 
nephanalysis  algorithms.  To  solve  this  problem,  the  contract  was  modified  to  add  two  new 
subtasks  to  provide  real-time  access  to  DMSP/OLS  and  NOAA/AVHRR  polar-orbiting 
satellite  data.  The  remaining  eight  tasks,  with  applicable  subtasks,  are  summarized  in 
Table  1. 

All  algorithms  developed  in  support  of  the  tasks  listed  in  Table  1  were  implemented 
for  testing  purposes  on  the  Air  Force  Interactive  Meteorological  System  (AIMS).  AIMS  is 
a  system  of  networked  general  purpose  computers,  image  processing  systems,  and  satellite 
ground  stations  maintained  at  the  Phillips  Laboratory  (Ivaldi  et  al.,  1992).  Documentation 
on  computer  programs  is  available  separately  in  the  required  TACNEPH  Software  Design 
Report.  All  software  development  for  TACNEPH  was  for  the  sole  purpose  of  testing 
algorithm  performance  under  real-world  conditions  using  actual  satellite  data  (see 
Section  10.1). 

Task  1  activities  were  designed  to  provide  a  suitable  environment  for  algorithm 
testing  by  simulating  the  database  capabilities  expected  to  be  available  on  a  tactical  terminal. 
These  were  primarily  directed  toward  satellite  sensor  data  ingest  and  management  but  also 
included  a  significant  effort  to  develop  a  visualization  capability  for  examining  algorithm 
internal  databases  and  results.  As  part  of  this  effort,  two  real-time  satellite  ground 
receiving  stations  to  ingest  direct-broadcast  transmissions  from  the  DMSP  FIO  and  FI  1 
and  NOAA-1 1  and  NOAA-12  satellites  were  installed  and  integrated  as  part  of  AIMS. 

An  SSM/I  all-weather  multiple  regression  algorithm  designed  to  estimate  brightness 
temperatures  that  would  be  measured  by  Sie  OLS  under  cloud-free  conditions  has  been 
used  at  AFGWC  in  the  RTNEPH  model.  Task  2  was  designed  to  evaluate  the  usefulness 
of  that  technique  in  the  TACNEPH  cloud  retrieval  algorithms.  Unfortunately,  regression 
coefficients  for  the  FIO  and  FI  1  satellites  could  not  be  supplied  to  the  TACNEPH  program 
during  the  period  of  performance  so  formal  testing  could  not  be  accomplished. 

Addressing  Task  3  requirements  comprised  the  principal  effort  under  TACNEPH. 
Each  subtask  was  a  major  research  effort  in  its  own  right  and  resulted  in  a  set  of  cloud 
algorithms  capable  of  analyzing  any  combination  of  sensor  channel  data  from  the  OLS  or 
AVHRR.  In  addition,  an  important  part  of  the  cloud  analysis  work  was  accomplished 
under  Task  3.3  where  algorithms  were  developed  to  predict  the  brightness  temperatures 
that  would  be  measured  by  the  long- wave  infrared  (LWIR)  chaimels  on  the  OL^  md 
AVHRR  under  cloud-free  conditions.  Accurate  dear-scene  temperature  estimates  are 
critical  to  OLS  cloud  algorithm  performance  and  also  have  a  significant  impact  on  AVHRR 
cloud  algorithm  accuracy. 

The  only  information  on  upper  air  characteristics  expected  to  be  available  on  the 
tactical  terminals  are  temperature  and  moisture  profiles  that  can  be  derived  from  satellite 
data.  TACNEPH  requirements  under  Task  4  specified  evaluation  of  a  government 
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Table  1.  TACNEPH  Task  Breakdown  Summary 


Task 

Name 

Description 

1. 

Tactical  Tenninal  Database 
Management 

Develop  capability  to  ingest,  calibrate,  Earth  locate,  process, 
and  display  databases  needed  for  TACNEPH 

1.1 

AIMS  database  management 

Modify  AIMS  database  manager  to  support  TACNEPH  files 

1.2 

AVHRR  data  acquisition 

Develop  software  to  ingest,  calibrate,  and  Earth  locate 

AVHRR  data 

1.3 

Satellite  ground  station 

Install  DMSP  and  NOAA  direct  broadcast  ground  stations 

1.4 

Real-time  data  ingest 

Develop  a  real-time  satellite  data  ingest  and  processing 
capability  for  DMSP  and  NOAA  sensor  data 

1.5 

Objective  analysis  of  point 
data 

Write  high-level  AIMS  application  to  generate  regular 
gridded  fields  from  unevenly  spaced  point  data 

1.6 

Image  processing 

Adapt  AIMS  image  processing  software  to  support 

TACNEPH  visualization  requirements 

2. 

SSM/I  surface  temperature 
retrieval 

Evaluate  impact  of  government  furnished  SSM/I  surface 
temperature  retrieval  algorithm  on  cloud  analysis  accuracy 

3. 

OLS  and  AVHRR  cloud  analysis 

Develop  cloud  and  surface  temperature  retrieval  algorithms 

3.1 

OLS  nephanalysis  algorithm 

Develop  OLS  cloud  detection  algorithm 

3.2 

AVHRR  nephanalysis 
algorithm 

Develop  AVHRR  cloud  detection  algorithm 

3.3 

OLS  and  AVHRR  surface 
temperature 

Develop  OLS  and  AVHRR  dear-scene  temperature  retrieval 
algorithm 

4. 

SSM/T  cloud  height  assignment 

Evaluate  utility  of  SSM/T  derived  vertical  temperature 
profiles  for  assignment  of  cloud  top  altitude 

5. 

Cloud  base  and  thickness 

Develop  improved  algorithm  for  estimating  cloud  thickness 
and  base  altitude  from  satellite  sensor  data  only 

6. 

Quality  control  and  tuning 

Investigate  QC  and  tuning  techniques  to  support  interactive 
manipulation  and  monitoring  of  cloud  analyses 

7. 

Conventional  data  processing 

Integrate  government  furnished  software  to  process  surface- 
based  cloud  observations  with  nephanalysis  results 

8. 

TACNEPH  computer  program 

Develop  software  to  implement  algorithms  developed  under 
other  tasks  for  test  and  validation  purposes 

furnished  SSM/T  temperature  profile  retrieval  computer  program  for  use  in  deriving  cloud 
top  altitude.  The  basis  for  this  evaluation  is  the  accuracy  of  die  retrieved  temperature 
profiles  relative  to  radiosonde  measurements. 

Task  5,  to  develop  algorithms  for  retrieval  of  cloud  thickness  and  base  altitude, 
differed  in  substance  from  the  other  TACNEPH  tasks  in  that  it  required  a  research  effort 
not  necessarily  intended  to  produce  an  operational  algorithm.  Estimation  of  cloud  optical 
properties  from  imaging  sensor  data  alone  (i.e.,  OLS  and  AVHRR)  is  problematic.  AER 
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derived  a  hybrid  technique  to  estimate  cloud  thickness  using  estimates  of  cloud  type  to  infer 
required  optical  characteristics.  Cloud  base  is  then  derived  using  the  thickness  and  cloud 
top  altitude. 

Accuracy  of  TACNEPH  derived  cloud  analyses  will  vary  with  the  quality  and  mix 
of  available  sensor  data  and,  potentially,  with  local  conditions.  Since  tactical  users  are 
unlikely  to  be  familiar  with  the  underlying  principals  of  the  TACNEPH  algorithms, 
additional  information  is  required  to  provide  estimates  of  the  relative  accuracy  of  a  given 
cloud  analysis.  It  was  also  necessary  to  develop  a  capability  to  adjust  algorithm  sensitivity 
to  particular  cloud  types  to  support  different  end-user  applications  and  location  s^ific 
conditions.  Task  6  addresses  these  requirements  by  providing  relative  accuracy  indicators, 
or  analysis  quality  flags,  as  a  by-product  of  the  cloud  analysis.  It  also  specifies  an 
interactive  tuning  capability  that  will  allow  the  user  to  modify  the  algorithm  characteristics 
to  satisfy  a  particular  application  with  no  knowledge  of  the  igoiithmic  approach. 

While  no  conventional  surface  or  aircraft  observations  of  cloud  are  available  on 
tactical  terminals,  it  is  likely  that  some  end  users  will  have  access  to  these  types  of  data 
from  other  external  sources.  Task  7  was  modified  during  the  program  from  the  initial 
requirement  of  simply  implementing  government  furnished  software  for  processing  of 
surface-based  observations  of  cloud  parameters  to  recommendations  on  how  to  combine 
TACNEPH  cloud  analysis  results  with  surface  observations  in  a  post-processing  sense. 

Possibly  the  single  most  important  aspect  of  the  entire  TACNEPH  program  was  the 
emphasis  throughout  on  real-data  testing.  It  is  worth  noting  that  while  there  are  a  large 
number  of  satellite  cloud  retrieval  techniques  reported  in  the  Uterature,  only  a  small  number 
have  been  tested  using  real  satellite  data,  and  of  those  most  are  based  on  results  from  only  a 
few  case  studies.  Even  during  initial  TACNEPH  algorithm  development,  testing  was 
based  on  real  satellite  data.  Case  study  data  were  collected  over  four  geographically  diverse 
locations  for  two  seasons,  winter  and  summer.  Later,  following  instdlation  of  the  DMSP 
and  NOAA  ground  stations,  real-data  testing  steadily  ^ew  into  what  became  a  daily 
process  that  entailed  automatic  analysis  of  every  satellite  pass  within  view  of  the  ground 
station  antennas  and  a  subsequent  evaluation  by  a  trained  analyst.  Feedback  from  the 
analyst  to  algorithm  developers  was  direct  and  resulted  in  numerous  improvements  and 
modifications  to  the  algorithms.  Ultimately  both  the  OLS  and  AVHRR  cloud  algorithms 
were  subjected  to  this  level  of  daily  review  for  one  full  year.  Task  8  requirements  provided 
the  framework  for  all  algorithm  testing  by  providing  a  full  functioned  enviromnent  for 
exercising  the  algorithms  over  a  broad  range  of  conditions  through  the  integration  of  data 
ingest,  database  management,  data  processing,  and  display  functions. 


2 .  External  Data  Requirements  and  Sources 

TACNEPH  nephanalysis  algorithms  are  designed  to  operate  on  satellite  data  from 
the  DMSP  OLS,  Special  Sensor  Microwave  Imager  (SSM/I)  and  Temperature  Sounder 
(SSM/T-1),  and  NOAA-AVHRR  sensors.  The  DMSP  and  NOAA  satelhtes  are  in  polar, 
sun-synchronous  orbits  whose  periods  are  approximately  104  minutes.  Data  from  DMSP- 
FIO  and  FI  1  and  NOAA-1 1  and  12  satellites  were  used  during  TACNEPH  algorithm 
development  and  testing.  Sensor  data  requirements  are  summarized  in  Table  2.  A  full 
discussion  of  the  sensor  data  including  instrument  characteristics,  sensor  calibration  and 
data  format  is  given  in  Sections  2.1  (visible),  2.2  (infrared),  and  2.3  (mission  sensor).  In 
addition  to  the  sensor  data,  TACNEPH  algorithms  require  five  supporting  databases:  Earth 
location  (Section  2.4),  sun-satellite  geometry  (Section  2.5),  surface  temperature  clima¬ 
tology,  (Section  2.6)  geography  classification  (Section  2.7)  and  lOOO-mb  height  field 
(Section  2.8).  These  databases  are  summarized  in  Table  3.  It  should  be  emphasized  that 
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Table  2.  Satellite  Sensor  Channel  Data  Attributes 


Satellite 

Sensor 

Channel 

(pm) 

Data 

Format 

Resolution^ 

(km) 

Bits  per 
Pixel^ 

Pixels  per 
Scan  Line^ 

NOAA 

AVHRR 

0.58-0.68 

percent  albedo 

1.1 

10 

2048 

0.72-1.10 

percent  albedo 

1.1 

10 

2048 

3.55-3.93 

EBBT^ 

1.1 

10 

2048 

10.3-11.3 

EBBT 

1.1 

10 

2048 

11.5-12.5 

EBBT 

1.1 

10 

2048 

DMSP 

OLS 

0.40-1.10 

counts 

2.7 

6 

1468 

10.5-12.6 

EBBT 

2.7 

8 

1468 

SSM/I 

19V  &  H  (GHz) 

EBBT 

25 

12 

64 

22V 

EBBT 

25 

12 

64 

37V  &H 

EBBT 

25 

12 

64 

85V  &H 

EBBT 

12.5 

12 

128 

SSM/T-1 

50.5  (GHz) 

EBBT 

214 

7 

53.2 

EBBT 

214 

7 

54.35 

EBBT 

214 

7 

54.9 

EBBT 

214 

7 

58.4 

EBBT 

214 

7 

58.825 

EBBT 

214 

7 

59.4 

EBBT 

214 

7 

'Sensor  resolution  at  satellite  subpoint. 

^AVHRR  radiance  data  are  transmitted  at  10-bit  resolution.  The  TACNEPH  development  system  could 
only  accommodate  8-bit  brightness  temperature  data;  however,  the  full  10-bit  resolution  is  used  in  the 
radiance  to  brightness  temperature  transformation. 

^Maximum  width  of  data  swath.  The  number  of  scan  lines  is  variable,  dependent  on  the  satellite-ground 
station  line  of  sight. 

^Equivalent  Blackbody  Brightness  Temperature. 


Table  3.  External  Supporting  Databases 


Type 

Resolution 

Projection 

Description 

Earth  Location 
Sun-satellite  Geometry 
Geography 

Surface  Temperature 
Climatology 

1000  mb  Height 

Profile 

10-Minute 
8th-Mesh 
(47  km) 
Whole-Mesh 

Scan 

Scan 

Lat-Lon 

Polar-Stereo 

Polar-Stereo 

latitude/longitude  for  each  pixel 

solar  zenith,  satellite  zenith,  azimuth  for  each  pixel 

ocean,  lake,  coast,  land,  desert 

monthly  climatology  at  3-hour  intervals  (Tdim) 

monthly  climatology  at  12-hour  intervals  (zdin^) 

TACNEPH  data  requirements  are  consistent  with  available  databases  on  tactical  terminals 
such  as  Mark  IV-B.  It  is  important  to  note  however,  that  database  resolution  and  projection 
information  provided  in  Table  2  are  provided  only  to  document  how  the  data  were  used  in 
the  TACNEPH  algorithm  development  work  and  should  not  be  interpreted  as  firm 
requirements.  TACNEPH  databases  were  constructed  solely  to  satisfy  research  needs  with 
no  attention  to  optimizing  data  access  or  processing  operations.  They  were  not  intended  to 
be  used  as  a  model  for  operational  implementation.  Actual  operational  data  formats  will  be 
dictated  by  the  host  tactical  terminal  database  capabilities  and  processing  environment.  For 
the  research  program  all  data  were  maintained  in  the  native  satellite  sensor  projection. 
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2.1  Visible  Sensor  Data 

The  majority  of  TACNEPH  cloud  analysis  algorithms  process  visible  reflectance 
data  in  a  relative  or  band-differencing  sense.  As  such,  visible  sensor  data  are  not  required 
to  be  absolutely  calibrated.  The  primary  reason  for  this  is  that  visible  counts  have  different 
physical  meanings  for  DMSP  and  NOAA  satellite  sensors.  For  OLS,  visible  counts  are 
proportional  to  upwelling  reflected  solar  energy  (Heacock,  1985).  For  AVHRR,  visible 
counts  are  linearly  proportional  to  percent  albedo  (Kidwell,  1988).  Thus,  to  more  precisely 
characterize  the  physical  meaning  of  each  data  source  the  following  naming  conventions  are 
adopted  for  this  report;  references  to  AVHRR  visible  data  are  termed  albedo,  while  all 
other  sources  of  visible  data  are  referred  to  as  visible  counts.  All  nephanalysis  algorithms 
described  in  this  document  expect  visible  satellite  sensor  data  to  conform  to  these 
conventions. 

2.1.1  AVHRR  Visible  Sensor  Data  Calibration 

AVHRR  visible  and  near-IR  data  were  calibrated  according  to  the  procedures 
published  by  Kidwell  (1988).  Sensor  counts  are  converted  to  a  percent  albedo  using  a 
hnear  function: 


Ai  =  SiX  +  Ii  (2-1) 

where  Ai  is  the  percent  albedo  for  charmel  i,  X  is  the  sensor  count  reported  in  the  telemetry 
data  stream,  and  Si  and  li  are  calibration  coefficients  established  prior  to  launch.  Percent 
albedo  is  defined  as  the  ratio  of  effective  scene  radiance  measured  by  the  sensor  to  the 
effective  radiance  that  would  be  measured  from  a  100%  reflecting  diffuse  surface  at  a  solar 
zenith  angle  of  0°.  To  convert  from  percent  albedo  to  radiance  the  albedo  values  are 
convolved  with  the  sensor  response  function.  This  is  done  numerically  as  follows: 


l=A, 


100  TCW 


(2-2) 


where  If  is  spectral  radiance  in  (W  m‘2  jim’l  sterl),  Fi  is  the  solar  radiance  integrated  over 
the  response  function  of  the  sensor  (W  m'2),  Wi  is  the  width  of  the  response  function 
(|xm),  and  i  indicates  AVHRR  channel  (1  or  2).  Tabular  values  for  the  AVHRR  visible  and 
near-IR  channel  calibration  and  sensor  response  coefficients  are  provided  by  Planet  (1988). 

OLS  visible  data  are  proportional  to  upwelling  reflected  solar  energy  and  are  not 
absolutely  calibrated  following  launch  (Heacock,  198^5). 


2.2  Infrared  Sensor  Data 

In  contrast  to  visible  sensor  data,  infrared  radiance  measurements  from  all  platforms 
are  required  to  be  absolutely  calibrated  and  subsequently  converted  to  equivalent  blackbody 
brightness  temperature  (EBBT).  Throughout  this  document  the  term  "brightness  tempera¬ 
ture"  is  treated  as  synonymous  with  EBBT.  Calibration  is  performed  differently  for  toe 
individual  sensors  but  generally  requires  a  linear  calibration  function  to  convert  IR  counts 
to  radiance.  Conversion  to  EBBT  is  then  performed  by  inverting  toe  Planck  function  over 
toe  bandpass- weighted  spectral  range  of  each  infrared  sensor  chaimel.  The  exception  to 
this  convention  is  toe  DMSP  OLS  sensor  which  performs  calibration  and  data  conversion 
onboard  toe  satellite  and  then  transmits  an  8-bit  IR  count  that  is  linearly  proportional  to 
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brightness  temperature.  Note  that  for  surfaces  where  the  blackbody  assumption  is  not 
correct  (i.e.,  emissivity  less  than  1.0),  computed  brightness  temperatures  can  be  signifi¬ 
cantly  different  than  the  physical  temperature  of  the  surface  being  viewed.  This  phenomena 
is  recognized  and  addressed  separately  by  the  individual  analysis  algorithms  (see  Sections 
5. 1  and  5.2),  and  can  be  useful  for  discriminating  between  different  types  of  clouds  and 
background  surfaces.  Other  factors  in  addition  to  surface  emissivity,  such  as  atmospheric 
attenuation,  can  also  result  in  satellite-derived  brightness  temperatures  that  are  different 
from  the  actual  temperature  of  the  viewed  surface.  These  conditions  are  addressed  in 
Section  4.2. 

2.2.1  AVHRR  Infrared  Sensor  Data 

AVHRR  mid  wave  and  long  wave  IR  data  were  calibrated  following  the  procedures 
of  Planet  (1988).  For  all  three  IR  channels  a  two-point  linear  calibration  is  performed  to 
convert  reported  sensor  counts  to  radiances  and  finally  to  equivalent  blackbody  brighmess 
temperatures.  The  response  of  the  channel  3  sensor  is  taken  to  be  linear  with  radiance  and 
the  two-point  calibration  is  sufficient;  however,  for  the  channel  4  and  5  detectors,  an 
additional  second-order,  nonlinear  calibration  step  is  required.  The  two-point  calibration  is 
performed  based  on  sensor  measurements  of  an  onboard  hot  source  and  cold  space  once 
every  five  AVHRR  HRPT  scans.  The  hot  source  is  monitored  by  four  redundant  platinum 
resistance  thermistors  (PRT).  Initially  it  is  necessary  to  convert  PRT  counts  to  temperature 
for  each  thermistor: 


Ti=Ea..X/  (2-3) 

j=0 

where  Ti  is  the  temperature  measured  by  thermistor  i,  ai  j  are  conversion  coefficients 
determined  from  a  pre-launch  calibration,  and  Xi  is  the  reported  count.  A  weighted  average 
of  the  four  thermistors  is  used  as  the  hot  source  reference  temperature: 

T  =  2b,T,  (2-4) 

i=l 

where  bi  is  the  weighting  coefficient  for  PRT  i,  also  determined  prior  to  launch.  To 
perform  the  calibration,  the  measured  hot  source  temperature  is  converted  to  radiance  by 
convoluting  the  Planck  function  with  the  sensor  response  function.  This  is  done 
numerically  through: 


X  b(v„T)  (})(v0  Av 
X  <l>(Vi)  Av 


(2-5) 


where  I  is  the  equiyalent  spectral  radiance  that  would  be  measured  from  a  blackbody  having 
a  temperature  of_T ,  B(vi,  T )  is  the  calculated  Planck  radiance  at  wave  number  v  for  a 
temperature  of  T ,  <j)(vi)  is  the  measured  response  function  of  the  sensor  at  wave  number 
Vi,  and  Av  is  a  wave  number  interval  suitable  for  the  numeric  integration.  In  practice  Vi  and 
(|)(vi)  are  provided  by  NESDIS  in  published  tables  (Planet,  1988).  Once  the  hot  source 
equivalent  radiance  has  been  computed,  the  linear  calibration  coefficients  (g  and  h)  for  each 
IR  channel  can  be  calculated  directly: 
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(2-6) 


X^-Xsp 

and 

h  =  Isp-gXsp  (2-7) 

where  Igp  is  radiance  from  deep  space  (3  K),  and  X,.  and  Xjp  ar^averag^sensor- 
measured  counts  for  the  hot  source  and  cold  space  respectively.  X.p  and  Xjp  are  averaged 
over  some  number  of  scan  lines  (generally  10)  to  minimize  the  effect  of  any  noise  in  I(T)  or 
the  sensor  measurements.  The  coefficients  are  applied  to  the  measured  count  for  each  pixel 
to  compute  a  scene  radiance: 


I(X)  =  gX  +  h.  (2-8) 

For  channel  3  data  the  scene  radiance  is  immediately  converted  to  a  brightness 
temperature  by  inverting  the  Planck  function. 

As  stated  above,  an  additional  second-order  nonlinear  correction  is  necessary  to 
correctly  compute  the  channel  4  and  5  radiances.  NOAA  provides  tables  for  the  correction 
values  based  on  laboratory  calibration  measurements  made  prior  to  launch  (Planet,  1988). 
Tabulated  values  are  provided  as  brightness  temperature  corrections  that  are  to  be  applied  to 
the  temperatures  obtained  from  the  first-order  c^ibration  above.  Correction  factors  are 
derived  from  a  quadratic  fit  to  the  instrament  calibration  response  at  10  K  intervals  over  the 
range  205-325  K  for  a  range  of  internal  hot  source  temperatures  between  10  and  20  C.  For 
TACNEPH,  the  non-linear  corrections  are  applied  by  first  converting  the  tabulated  tempera¬ 
ture  values  back  to  radiance  using  Eq.  2-5  and  then  fitting  a  quadratic  to  the  resultant 
values.  The  non-linear  corrections  are  applied  to  the  channel  4  and  5  radiances  obtained 
from  Eq.  2-8,  and  the  corrected  radiances  are  then  converted  to  brightness  temperatures. 

2.3  Mission  Sensor  Data 

In  addition  to  imagery  channel  data  described  above  in  Section  2.2,  TACNEPH 
requires  use  of  microwave  data  from  DMSP  SSM/I  and  SSM/T  mission  sensors. 

2.3.1  SSM/I 

The  SSM/I  is  a  passive  microwave  imager  used  to  estimate  various  geophysical 
properties  of  the  Earth  and  atmosphere  including  surface  skin  temperature,  ice  and  snow 
cover,  soU  moisture,  precipitation,  and  precipitable  water.  It  has  seven  channels:  19V, 
19H,  22V,  37V,  37H,  85V  and  85H  GHz.  The  SSM/I  uses  a  conical  scan  which  results  in 
footprints  that  have  the  same  spatial  dimensions  across  the  scan  for  a  given  frequency. 

2.3.2  SSM/T-1 

The  SSM/T-1  is  a  passive  microwave  temperature  sounder  used  primarily  to 
provide  temperature  and  pressure  height  profiles  of  the  atmosphere.  T-1  also  has  seven 
channels:  50.5,  53.2,  54.35,  54.9,  58.4,  58.825,  and  59.4  GHz.  The  50.5  GHz  channel 
is  an  atmospheric  window  channel.  Each  subsequent  channel  has  greater  sensitivity  to 
higher  altitudes  in  the  atmosphere,  up  to  30  km  for  the  59.4  GHz  channel. 


8 


2.4  Earth  Location 


The  TACNEPH  database  was  set  up  such  that  orbital  parameters  are  not  maintained 
for  each  satellite  scene  (in  some  cases,  depending  on  the  source  of  the  satellite  data,  they 
were  not  even  available).  Therefore,  to  provide  the  required  Earth  location  information  it  is 
necessary  to  maintain  separate  latitude-longitude  (LL)  files  for  each  scene.  These  files  are 
produced  automatically  during  sensor  data  ingest  and  are  maintained  in  the  same  spatial 
projection  as  the  sensor  data  but  at  degraded  spatial  resolution.  A  linear  run-time  interpola¬ 
tion  technique  is  used  to  access  the  LL  files  and  calculate  the  location  of  each  pixel.  For  the 
operational  implementation  of  the  program  the  only  requirement  is  to  provide  Earth  location 
information  on  demand  for  each  analysis  location.  This  requirement  holds  regardless  of 
whether  the  data  are  pre-registered  to  a  standard  grid  or  left  in  scan  projection.  In  either 
case  it  is  not  necessary  to  duplicate  the  LL  file  scheme  used  in  the  development  progrtun. 

2.5  Sun  -  Satellite  Geometry 

Similar  to  the  Earth  location  information  described  above,  satellite  and  solar  zenith 
angle  and  sun-satellite  azimuth  angle  data  must  also  be  provided  for  each  analysis  point. 
Figure  1  provides  a  schematic  representation  of  the  angle  definitions.  These  angles  are 
used  to  classify  solar  illumination  conditions  and  to  characterize  sun  glint  in  visible 
reflectance  data.  Like  the  LL  files,  the  angle  data  are  maintained  in  separate  files  generated 
during  the  sensor  data  ingest  operation.  They  are  also  carried  at  reduced  spatial  resolution 
and  are  retrieved  using  a  linear  run-time  interpolation  routine.  Again,  the  only  requirement 
that  affects  the  operational  implementation  is  a  capability  to  retrieve  the  angle  data  for  each 
pixel  on  demand. 


Zenith 


point 


Figure  1.  Satellite-earth-solar  geometry  (after  Taylor  and  Stowe,  1984); 

W- satellite  zenith  angle,  0  -  solar  zenith  angle,  and  <P  -  azimuth  angle. 
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2 . 6  Climatology  Databases 

Information  on  the  radiating  temperature  of  the  terrestrial  background  is  needed  for 
cloud  detection  schemes  and  background  characterization.  The  only  database  available  on 
the  Mark  IV-B  that  provides  any  information  on  surface  temperature  is  a  global  "surface 
temperature"  cUmatology.  For  the  TACNEPH  program,  a  surface  temperature  database 
was  obtained  from  the  Air  Force  Environmental  Technical  Applications  Center  (ETAC)  that 
is  believed  to  be  the  same  as  that  available  in  the  Mark  IV-B  database.  The  information  we 
have  received  about  this  database  is  that  it  was  compiled  from  RTNEPH  surface 
temperature  data  over  a  multi-year  period  and  is  representative  of  shelter,  rather  than  skin, 
temperatures  (personal  communication,  James  T.  Bunting).  These  data  are  maintained  at  a 
resolution  of  0.1  K  and  are  comprised  of  monthly  average  temperatures  at  three-hour 
intervals  (i.e.,  0,  3,  6,  9,  12, 15,  18,  and  21  UTC)  stored  on  an  8th-mesh  grid. 
Temperature  data  are  retrieved  from  the  database  by  identifying  the  8th-mesh  grid  box  that 
contains  the  target  pixel  and  then  time  interpolating  to  match  the  observed  sensor  data. 
Additional  information  on  the  use  of  surface  temperature  data  is  provided  in  Section  4.2. 

2 . 7  Geography  Database 

Geography  data  are  needed  to  characterize  the  background  conditions  for  several  of 
the  cloud  detection  tests.  Of  particular  importance  is  the  boundary  between  land  and  water 
backgrounds.  Highly  reflective  surfaces  such  as  snow,  ice,  desert,  and  sun  glint  also  have 
significant  impact  on  any  tests  that  rely  on  measurements  of  reflected  solar  energy  for  a 
signature.  The  geography  database  developed  for  TACNEPH  provides  a  characterization 
of  the  background  type  at  10  arc-minute  resolution.  Background  types  include  ocean,  lake, 
land,  coast  and  desert.  While  it  is  recognized  that  this  database  does  not  currently  exist  on 
the  Mark  IV-B,  it  could  easily  be  made  available  and  should  be  considered  for  transition 
along  with  the  TACNEPH  algorithms.  If  available  a  higher  resolution  database 
approaching  satelhte  sensor  resolution,  with  specific  emphasis  on  accurate  placement  of 
coastlines,  would  significantly  improve  the  overall  cloud  analysis. 

2.8  1000-mb  Height  Profile  Database 

A  1000-mb  height  database  is  required  for  the  retrieval  of  atmospheric  temperature 
height  profiles  using  SSM/T-1  microwave  sounder  data.  This  profile  is  in  turn  used  to 
specify  cloud  height.  The  1000-mb  database  was  available  for  development  purposes  as 
part  of  the  Sea  Space  DMSP  ground  station  system  on  site  at  the  Phillips  Laboratory. 

3.  TACNEPH  Database 

Database  related  work  (Task  1,  Table  1)  was  a  major  effort  during  TACNEPH. 

The  basic  requirement  was  to  emulate  Ae  relevant  database  capabihties  that  would  be 
available  in  a  tactical  terminal,  particularly  as  they  relate  to  satellite  sensor  data.  This  was 
necess^  to  handle  the  large  amount  of  data  required  to  support  algorithm  testing  and 
validation  (see  Section  5.3).  All  work  was  performed  on  the  AIMS  computer  system 
which  had  only  rudiment^  satellite  database  capabilities  at  the  start  of  the  program. 
Database  management  as  it  applied  to  TACNEPH  involved  three  separate  functions: 

1)  data  ingest  (data  unpacking,  calibration.  Earth  location); 

2)  database  organization  (query,  retrieve,  add,  modify,  delete);  and 

3)  data  visualization  (image  processing,  interactive  evduation  and  testing,  and 
synthetic  image  rendering). 
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Data  ingest  was  handled  on  an  ad  hoc  basis  for  each  separate  source  of  data  and  as 
such  is  not  generally  applicable  and  will  not  be  detailed  here.  However,  there  has  been 
considerable  interest  in  the  database  management  and  visualization  functions  expressed  at 
numerous  TACNEPH  program  reviews,  so  descriptions  of  these  functions  are  provided. 

3.1  AIMS  Database  Management 

AIMS  database  management  work  was  performed  under  Task  1.1  (Table  1).  The 
goal  of  this  task  was  to  modify  existing,  and  to  develop  new,  AIMS  database  management 
software  to  support  and  maintain  the  TACNEPH  data  file  suite.  For  this  task  it  was  first 
necessary  to  specify  the  database  requirements  in  terms  of  the  types  and  volume  of  data  that 
needed  to  be  handled,  how  the  data  were  to  be  used  at  the  user-application  level,  how  and 
where  to  store  it,  and  how  to  optimize  data  access  while  minimizing  the  impact  on  user 
applications. 

Three  major  types  of  satellite  data  are  of  concern  to  the  TACNEPH  project.  The 
foremost  is  satellite  imagery,  encompassing  raw  sensor  data  measured  by  visible,  infrared, 
or  microwave  detectors  onboard  an  orbiting  satellite.  The  second  is  imagery-supporting 
data,  including  Earth  location,  sensor  calibration,  and  viewing  geometry  information. 

These  data  compliment  the  imagery  data  by  providing  ancillary  information  that  is  known  at 
data  collection  time  but  not  directly  stored  or  encoded  with  the  imagery  data.  The  third  data 
type  is  analysis  data.  Generally,  when  sensor  data  are  processed  t&ough  a  cloud  analysis 
dgorithm,  pixel  resolution  product  data  are  generated.  As  an  example,  a  given  analysis 
data  file  might  contain  information  describing  which  pixels  of  a  given  multi-channel  scene 
are  classified  as  cloudy.  Another  analysis  might  form  a  more  generalized  result,  or  only 
represent  subsets  of  imagery  data.  The  results  for  these  and  other  user-specified  analysis 
products  compliment  the  imagery  data  in  the  same  manner  that  the  supporting  ancillary  data 
do  and,  as  such,  are  logically  tied  to  the  input  imagery  database  files  by  the  database 
manager. 

For  TACNEPH  a  database  management  strategy  was  developed  that  addresses 
multiple  requirements.  The  amount  of  data  handled  in  the  TACNEPH  effort  is  large.  Data 
from  one  pass  of  a  multi-channel  polar-orbiting  satellite  can  easily  occupy  30  to  40 
megabytes  of  storage  space,  even  before  any  ancillary  or  analysis  data  have  been  generated. 
A  unique  unit  of  data  had  to  be  defined  such  that  accessing  of  data  by  user  applications 
would  not  cause  conflicts  or  aliasing  with  other  data  in  the  database.  The  storage  strategy 
had  to  support  migration  of  data  from  one  physical  media  to  another.  Migration  of  data 
implies  that  the  user  could  move  the  actual  location  of  the  raw  data  sets  (for,  say,  space 
allocation  concerns)  without  any  impact  on  the  functionality  of  applications  that  use  the 
data.  A  method  for  storage  and  retrieval  of  data  from  the  entire  database  was  engineered 
such  that  both  permanent  and  removable  media  could  be  used  in  a  manner  transparent  to 
any  user  applications.  The  removable  media  encompasses  the  same  file  structure  as  the 
permanent  storage  hardware  currently  residing  with  the  AIMS  systems. 

Once  a  strategy  for  storing  data  was  established,  a  method  for  accessing  the  data 
was  devised.  Providing  the  applications  user  with  a  list  of  all  data-directories  and  disks 
containing  information  about  what  sensor  data  is  available  places  a  heavy  burden  on  that 
user.  A  linearly  organized  list  of  all  raw  data  sets  is  a  minimal  asset  when  the  data  list  is 
large.  In  order  to  manage  the  data  as  a  resource  shared  by  many  users  at  all  levels  of 
investigation  and  research,  it  became  essential  that  information  about  the  data  be  clearly 
specified  and  defined,  easily  accessible,  and  well-controlled. 
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There  is  much  ancillary  and  inferred  data  that  describes  imagery  data  such  as  the 
observing  satellite,  location,  time,  and  other  information  that  can  assist  a  user  to  identify 
and  access  required  data.  Metadata  are  a  specialized  type  of  information  used  to  describe 
large  data  sets  that  can  be  encoded  in  a  compact  computer-readable  form  for  optimized 
storage  processing  and  end  user  availability.  A  series  of  metadata  databases  were 
developed  to  describe  the  large  volumes  of  raw  data  for  the  TACNEPH. 

Two  software  products  were  designed,  developed,  and  implemented  in  accordance 
with  the  above  requirements.  They  consist  of  a  low-level  user  library  of  metadata 
management  routines,  and  a  high-level  metadata  scanning  utility.  A  software  library 
consisting  of  numerous  database  management  routines  was  developed,  and  fully 
implemented  on  the  AIMS  cluster.  These  routines  operate  on  metadata  that  describe  the 
contents  of  the  entire  TACNEPH  data  set.  The  routines  are  low-level,  that  is,  a  user  may 
call  any  of  these  modules  from  a  variety  of  applications.  Each  low-level  routine  has  a 
single,  well  defined  function.  Status  codes  are  used  to  report  the  success  or  failure  of 
specific  operations  and  are  the  only  control  information  passed  back  to  the  calling 
application.  As  such  these  low-level  routines  do  not  exert  control  over  the  operation  of  a 
calling  application. 

The  high-level  utility  was  developed  to  perform  search  and  retrieve  operations  on 
the  combined  metadata  databases  from  the  command  line.  This  utility  serves  as  a  fi-ont-end 
product  to  the  applications  user,  in  many  cases  eliminating  the  need  for  application  specific 
software  development.  It  is  used  to  search  and  sort  through  metadata  records  from  some  or 
all  metadata  databases  for  information  about  specific  records,  or  groups  or  records  to 
retrieve  information  on  database  entries  that  match  specified  criteria. 

3.1.1  Database  Management  System  Design 

The  TACNEPH  database  management  system  (TDB)  was  designed  to  satisfy  both 
data-storage  and  relational  data-management  requirements.  In  addition  to  the  project- 
imposed  requirements  discussed  in  Section  3.1,  the  AIMS  computer  systems  imposed  their 
own  set  of  requirements  on  the  database  design.  Left  unchecked,  the  volume  of  data 
represented  by  the  TACNEPH  data  set  would  overwhelm  AIMS  resources  and  make  it 
unusable  to  the  other  research  efforts  concurrently  dependent  on  this  system.  Further,  the 
TACNEPH  data  set  is  dynamic  and  open-ended  forcing  the  database  management  strategy 
to  incorporate  expansion  and  adaptability  to  increasing  volumes  of  data,  especially  as  those 
volumes  increase  rapidly.  To  address  these  issues  a  requirement  that  the  database 
implementation  allow  for  removable  storage  media  was  imposed.  This  requirement  was 
satisfied  via  the  use  of  optical  disks. 

The  data  to  be  used  for  the  TACNEPH  project  is  scan-formatted  multi-channel 
satellite  imagery  with  supporting  Earth  location  ancillary  data.  To  facilitate  processing  and 
exploit  the  natural  organization  of  the  sensor  data,  all  data  are  maintained  in  scan  line 
format.  One  file  is  created  for  each  imager  charmel,  another  for  earth-location  data,  and 
another  for  additional  scan  geometry  information.  For  an  average  case  which  covers 
roughly  15  minutes  of  high-resolution  coverage,  one  set  of  files  occupies  approximately  35 
megabytes  of  disk  space.  Supporting  data  are  stored  in  formats  specifically  developed  for 
TACNEPH  applications. 

In  considering  the  problem  of  how  to  physically  store  all  types  of  TACNEPH  data, 
a  single  data  entity  that  conceptualized  the  smallest  common  denominator  for  aU  data  types 
had  to  be  defined.  That  unit  was  determined  to  be  the  case-study  day.  With  this  definition 
of  a  data  unit,  a  method  for  the  actual  storage  of  data  evolved.  The  method  is  as  follows: 


under  a  generic  disk  name  a  directory  structure  is  created  that  contains  a  series  of  subdirec¬ 
tories.  These  subdirectories  are  named  in  accordance  with  the  Julian  date  corresponding  to 
the  valid  time  of  the  satellite  data.  The  files  contained  within  these  subdirectories  are 
uniquely  named  to  reflect  both  the  origin  of  the  data  (i.e.,  which  satellite  and  sensor 
collected  the  data)  and  the  site  over  which  the  data  was  taken.  An  example  of  a  directory 
listing  for  a  TACNEPH  data  subdirectory  is  provided  below: 

Directory  ofTDB$DISK:[TACNEPH.DATA.<YYYDDD>]: 

N12_001_JAM_121_13.DAT 

N12_002_JAM_121_13.DAT 

N12_003_JAM_121_13.DAT 

N12_004_JAM_121_13.DAT 

N12_005_JAM_121_13.DAT 

N12_LAT_JAM_121_13.DAT 

N12_ANG_JAM_121_13.DAT 

N12_001_ALB_121_13.DAT 

N12_002_ALB_121_13.DAT 

N12_003_ALB_121_13.DAT 

N12_004_ALB_121_13.DAT 

N12_005_ALB_12 1_13.DAT 

N12_LAT_ALB_121_13.DAT 

N12_ANG_ALB_121_13.DAT 

Note  that  in  the  example  ‘TDB$DISK”  is  a  logical  definition  pointing  to  the  specific 
disk  on  which  the  data  directory  structure  resides.  TACNEPH.DATA  is  the  generic  data 
directory  stmcture  and  <YYYDDD>  is  the  date  of  the  observation.  The  file  names  are 
unique  mnemonics  that  not  only  identify  the  file,  but  the  satellite,  data  type,  location,  date 
and  time  of  observation.  This  ^lows  for  multiple  cases  observed  during  a  single  day  to  be 
stored  in  the  same  directory,  uniquely.  In  this  sample  directory  listing,  there  are  files  from 
two  separate  data  runs,  taken  on  the  same  day,  but  over  different  locations. 

The  disk  name  and  data  directory  structure  are  important  to  execution  of  the  second 
part  of  the  storage  strategy.  Under  VMS,  a  disk  name  is  a  logical  representation  of  an 
actual  device.  If  this  device  is  formatted  in  the  Digital  Equipment  Corporation  (DEC) 

Files- 1 1  standard,  then  the  device  can  be  accessed  in  a  consistent  way  independent  of  the 
type  of  medium  used.  At  the  beginning  of  this  project,  several  erasable  optical  disk  drives 
were  installed  on  the  AIMS  cluster  to  exploit  this  format  convention.  The  optical  drives  use 
removable  disks  that  store  600  megabytes  of  data,  and  can  perform  access  operations  at 
speeds  comparable  to  those  of  the  permanent  hard  drives  installed  on  the  cluster.  Using 
logical  names  instead  of  physical  disk  names,  and  adhering  to  the  Files- 1 1  standard, 
provides  a  method  for  implementing  a  generic  storage  strategy  over  multiple  media  types, 
independent  of  their  type  (e.g.,  removable  vs.  non-removable),  that  is  invisible  to  the 
applications  user.  Supporting  software  was  developed  to  inform  the  user  whether  loading 
an  optical  disk  is  necessary  to  ran  an  application.  Everything  else  about  the  storage 
structure  (e.g.,  data  sub-directory)  is  independent  on  the  actual  location  of  the  data  and  thus 
transparent  to  the  user. 

3.1.2  Database  Structure 

The  next  stage  of  the  design  addressed  data  access.  This  was  solved  through  use  of 
a  relational  metadata  database  model.  The  management  of  all  the  data  sets  of  the  types 
described  in  Section  3.1  is  handled  by  a  collection  of  metadata  files.  All  data  management 
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software  acts  only  on  the  metadata  files  which  are  maintained  separately  from  the  sensor 
and  supporting  data  files. 

Recall  that  metadata  describes  a  specific  class  of  data  that  are  used  to  describe  other 
data.  Typically,  metadata  records  are  much  smaller  than  the  units  of  data  they  describe. 

For  example,  a  satellite  image  requires  tens  of  megabytes,  however,  the  information  that 
describes  the  data  (e.g.,  location,  time,  satellite...)  typically  only  requires  a  few  tens,  or  at 
most  hundreds,  of  bytes.  For  TDB,  metadata  files  are  organized  based  on  the  information 
content  inherent  in  each  data  set.  For  example,  separate  metadata  files  were  created  to 
describe  imagery  and  supporting  data.  The  metadata  are  permanently  associated  with  the 
data  file  they  describe  by  the  database  management  software.  Supporting  data  are  linked  to 
the  appropriate  imagery  data  by  relations  inherent  in  the  metadata  entries. 

When  gathering  metadata  entities  to  be  used  in  a  data  management  strategy,  the 
organization  of  these  entities  becomes  important.  Randomly  organized  metadata  are  of  no 
more  use  than  randomly  organized  raw  data  sets.  For  the  TACNEPH  application,  the 
organization  of  metadata  entities  into  conceptual  units  was  necessary.  These  units  are 
referred  to  as  (meta)  data  dictionaries.  For  TACNEPH,  satelhte  imagery  and  supporting 
data  are  described  by  metadata  entries  in  a  data  dictionary.  Each  set  of  entries  describes  one 
database  entry  (i.e.,  one  data  file),  and  each  dictionary  describes  one  type  of  data  (e.g., 
imagery,  supporting).  In  the  TACNEPH  implementation  of  this  design,  the  data 
dictionaries  are  themselves  organized  in  a  relational  maimer,  thus  providing  the  logical  links 
between  different  types  of  data.  Each  of  the  data-dictionaries  are  described  in  brief  below; 

The  CORE  data-dictionary  contains  the  minimal  set  of  metadata  elements  which  can 
describe  every  physical  data  object  (i.e.  image  file,  analysis  file)  registered  in  the 
TACNEPH  database.  Its  primary  function  is  to  provide  a  link  to  every  piece  of  data  in  the 
TACNEPH  database. 

The  S  ATIMG  data-dictionary  contains  metadata  elements  which  describe  imagery 
files  in  the  TACNEPH  database.  Parameters  such  as  number  of  rows,  number  of  columns, 
and  number  of  bit-planes  are  stored  in  these  meta-records  with  reference  links  to  associated 
non-imagery  data  objects  in  the  TACNEPH  database. 

The  ANGUES  and  LATLON  data-dictionaries  contain  metadata  elements  which 
describe  scene/satellite-geometry  and  geopositional  files  in  the  TACNEPH  database. 
Parameters  such  as  satellite  orientation,  sensor  footprint,  and  subsampling  resolution  are 
stored  in  these  meta-records  with  reference  links  to  associated  non-angle^non-latlon  data 
objects  in  the  TACNEPH  database. 

The  PRODUCTS  data-dictionary  contains  metadata  elements  which  describe  analy¬ 
sis  files  in  the  TACNEPH  database.  The  parameters  in  these  dictionaries  vary  depending 
on  the  type  of  analysis  being  tracked  by  the  database.  Reference  links  to  all  the  associated 
raw  data  (imagery,  angles,  latlon,  etc.)  used  in  generating  the  analysis  are  also  tracked. 

The  COMMENTS  data-dictionary  contains  subjective  user-comments  associated 
with  a  single  entry  in  the  TACNEPH  database.  There  is  no  limit  to  the  number  of 
individual  comments  that  can  be  tracked  by  the  TACNEPH  database  software. 

The  CORE  data-dictionary  contains  entries  for  every  data  object  in  the  TACNEPH 
database.  The  other  data-dictionaries  contain  entries  for  unique  sub-sections  of  the 
database,  but  are  linked  with  associated  database  entries  such  that  a  query  made  on  any  of 
these  data-dictionaries  will  guide  the  retrieval  application  to  the  appropriate  data.  The 
power  of  this  design  becomes  apparent  when  one  considers  the  problem  of  accessing  a 
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specific  data  file.  Assume  one  data  file  occupies  30  MB  and  the  database  contains  100 
images  of  this  size  to  track.  A  metadata  database-record  that  contains  information  about 
one  of  these  images  would  require  no  more  than  a  few  hundred  bytes.  A  metadata  database 
containing  records  for  100  images  would  require  only  a  few  kilobytes  of  storage  space.  It 
is  likely  that  even  a  purely  random  search  of  the  mettles  would  be  fast  enough  to  satisfy 
most  applications  while  any  type  of  search  of  the  raw  data  files  would  probably  be 
prohibitively  long.  This  becomes  even  more  apparent  given  the  distributed  nature  of  the 
TACNEPH  data  set.  The  TACNEPH  database  stracture  is  a  collection  of  organized 
metadata  data-dictionary  files.  All  data  dictionaries  are  maintained  on-line  in  a  centralized 
location  while  the  data  files  themselves  are  distributed  and  can  be  located  anywhere 
including  off-line  media.  Optimization  of  the  search  and  access  process  is  attained  through 
organizing  the  data  dictionaries  based  on  the  most  common  query  parameters. 

The  TACNEPH  database  functions  on  a  one-to-one  (or  linear)  relational  model.  The 
linear  model  offers  uniqueness  such  that  every  member  of  the  data  set  is  represented  by  a 
separate,  mdimentary  data  dictionary  entry,  hi  the  TACNEPH  application,  there  are  various 
classes  of  raw  data  sets  to  be  organized.  Each  of  these  can  be  described  by  metadata,  but 
results  in  a  collection  of  metadata  groups  that  themselves  require  an  organization  strategy  to 
optimizes  their  usefulness.  A  database  management  design  that  enhances  the  robustness  of 
a  (single)  relational  metadata  model  was  sought  to  impose  a  structure  on  the  evolving  groups 
of  metadata.  This  was  accomplished  through  the  introduction  of  a  high  level  core  data 
dictionary  that  provides  the  relationships  between  the  lower  level  dictionaries. 

The  "atomic-unit"  of  the  TACNEPH  data  set  is  the  individual  data  file.  The  atomic- 
unit  of  the  TACNEPH  database  is  an  individual  core  data-dictionary  meta  record.  The  core 
dictionary  file  contains  entries  for  all  data  files  in  the  database.  Each  entry  has  a  unique 
entry  number  and  contains  metadata  describing  the  I/O  characteristics  of  a  single  data  file 
independent  of  type.  Lower  level  dictionaries  (i.e.,  SATIMG,  LATLON,  ANGLES, 
PRODUCT,  COMMENT)  describe  data-specific  characteristics  of  only  one  type  of  data  file 
(imagery,  earth  location,  satellite  geometry,  analysis  data).  For  example,  sensor  data 
obtained  from  a  single  OLS  pass  generates  a  number  of  entries  in  different  data 
dictionaries.  Five  entries  in  the  core  dictionary  are  made  since  there  are  five  data  files  from 
this  pass  to  track  (visible  and  IR  sensor  chaimel  data  plus  LATLON,  ANGLES,  and 
PRODUCT  data).  The  visible  and  infrared  data  are  described  by  two  entries  in  the 
SATIMG  dictionaty.  The  metadata  which  describe  this  pass  in  the  SATIMG  dictionary 
include:  the  satellite  and  sensor  type,  the  number  of  rows  and  columns  in  the  image,  the 
image  resolution  (km/pixel),  offset  information  for  mapping  software,  and  time  stamps  for 
both  the  time  of  ingest  and  time  of  orbital  pass.  Earth  location  and  scan  geometry  data 
associated  with  the  pass  are  described  in  the  LATLON  and  ANGLES  dictionaries 
respectively.  It  should  be  noted  that  unlike  imagery  data  which  are  stored  at  pixel-level 
resolution,  earth-location  and  satellite  geometry  data  are  not  required  at  such  high  resolution 
and  thus  are  stored  at  a  sub-sampled  resolution  to  minimize  mass  storage  requirements. 

The  sampling  frequency  varies  with  the  satellite  type.  Sub-sampled  data  are  interpolated  at 
mn  time  when  pixel-level  data  retrievals  are  processed.  The  impact  on  processing 
performance  and  positional/geometric  accuracy  has  been  minimal.  Metadata  in  these 
dictionaries  include  information  on  the  sampling  frequency  of  the  data,  and  any  positional 
shift  in  the  data  imposed  on  interpolation  routines. 

Derived  analyses  files  are  described  in  the  product  dictionary.  The  metadata  in  this 
dictionary  describe  characteristics  that  are  similar  to  the  SATIMG  and  core  dictionaries,  but 
also  take  into  account  the  multiple  occurrence  of  analyses  per  single  data  entry  in  the  other 
data  dictionaries.  User  comments  can  also  be  registered  about  this  pass  in  a  comment 
dictionary.  All  of  these  data  files  are  also  registered  in  the  core  data  dictionary.  One 
dictionary  is  distinguished  from  another  by  tiie  information  they  contain. 
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The  core  dictionary  provides  pointers  between  the  various  data  dictionary  entries 
that  are  all  related  to  the  same  pass.  For  the  TACNEPH  application,  this  is  a  very  powerful 
design  for  several  reasons.  When  searching  through  multiple  related  data  dictionary  entries 
only  limited  information  on  one  specific  data  type  can  be  obtained  from  a  single  data  dic¬ 
tionary.  Complete  descriptions  of  the  entire  related  data  set  can  only  be  obtained  by 
accessing  all  related  metadata  dictionaries.  To  facilitate  this  process  all  data  dictionary  files 
are  organized  using  a  file  structure  known  as  Indexed  Sequential  Access  Method  (ISAM). 
ISAM  provides  a  format  for  rapidly  accessing  individual  entries  based  on  a  specified 
descriptor  known  as  a  key.  In  TDB,  each  database  entry  is  assigned  a  unique  entry 
number.  In  all  cases  the  dictionary  files  use  the  entry  number  as  the  primary  key.  For  a 
given  set  of  related  data  files,  the  core  dictionary  maintains  the  entry  numbers  for  all  related 
entries  in  the  various  lower  level  dictionaries.  Thus,  from  any  data  dictionary,  access  to 
any  other  related  dictionary  entry  is  straight-forward  and  direct  through  a  query  of  the  core 
dictionary  for  the  related  entry  number.  Keyed  access  of  data  is  a  property  of  the  VMS 
operating  system  that  is  handled  well.  The  combination  of  robust  design  and  the  existence 
of  VMS  tools  that  allow  its  easy  implementation  made  this  solution  an  attractive  one  for 
TACNEPH. 


3.1.3  TACNEPH  Database  Implementation 

The  TACNEPH  database  was  successfully  implemented  on  AIMS  and  used 
extensively  for  several  years  to  handle  both  real-time  and  retrospective  data.  All  software 
was  developed  using  the  required  VMS  FORTRAN-77  programming  language  and  making 
extensive  use  of  VMS  specific  utilities  and  programming  libraries.  A  hierarchy  of  data 
dictionaries  was  used  to  organize  the  descriptor  data  by  category.  Table  4  provides  a 
description  of  the  data  dictionaries.  Metadata  dictionary  files  are  organized  into  primary 
and  secondary  databases.  A  primary  metadata  database  is  one  that  contains  unique  indexed 
entries  related  to  other  data  dictionaries  along  structured  hierarchical  lines  which  converge 
at  the  master  (core)  database.  A  primary  descriptor  entry  would  describe  attributes  for  only 
one  entry  in  the  core  database,  and  thus  is  distinguishable  along  the  primary  index  of  the 
core  data  dictionary.  A  metadata  database  of  secondary  design  is  one  that  contains  non¬ 
unique  (related)  indexed  entries  that  may  be  associated  to  multiple  other  databases  (primary 
or  secondary).  An  example  of  a  secondary  database  is  a  user  comment  database.  Multiple 
comments  might  describe  a  common  core  database  entry,  and  thus  are  not  distinguishable 
along  the  core  database’s  primary  index. 

Table  4.  TACNEPH  Database  Data  Dictionary  Descriptions 


Name 

Description 

CORE 

Contains  list  of  related  data  dictionary  entries  and  file  access  information 

SATIMG 

LATLON 

ANGLES 

Describes  imageiy  files  generated  from  satellite  sensor  data 

Describes  files  containing  Earth  location  information 

Describes  files  containing  satellite  scan  and  solar  geometry  information 

COMMENT 

Contains  anecdotal  information  provided  by  research  scientists  and  users 

PRODUCT 

Describes  files  containing  analysis  algorithm  results 

In  the  TACNEPH  database  application,  the  structure  of  the  primary  and  secondary 
databases  serves  to  order  and  control  Ae  organization  of  metadata.  The  structural  hierarchy 
of  the  primary  databases  defines  the  data  flow  control  for  search  and  retrieval  algorithms. 
This  structure  orders  the  complete  metadata  about  a  given  data  object  by  most  important  and 
relevant  qualities.  The  non-hiierarchical  structures  between  secondary  and  primary 
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databases  act  as  operators  on  the  primary  metadata.  Secondary  data  are  incomplete  by 
themselves  but  can  operate  on  any  primary  database  anywhere  within  the  hierarchy  tree  and 
augment  the  information  content  of  the  primary  metadata.  For  example,  when  a  user  com¬ 
ment  about  a  particular  image  is  accessed,  it  signifies  relatively  little  objective  information. 
However,  when  combined  with  a  related  primary  metadata  field,  the  comment  augments 
that  objective  information  by  providing  a  subjective  depth.  As  such,  secondary  databases 
may  operate  on  any  level  of  primary  metadata,  but  provide  little  or  no  objective  information 
by  themselves. 


3.1.4  TACNEPH  Database  Summary 

Requirements  for  the  TACNEPH  database  management  task  were  satisfied  through 
the  development  of  a  dynamic,  relational  database  management  and  storage  capability;  a 
low-level  database-management  software  library;  and  a  high-level  utility  program.  The 
relational  database  and  its  associated  management  strategy  optimally  represent  and  control 
large  sets  of  meteorological  data,  and  support  database  growth  as  a  part  of  its  design.  The 
low-level  software  library  is  a  collection  of  independent  modules  that  manage  the 
TACNEPH  metadata  data  dictionary  files.  The  high-level  application  program  is  a  user- 
interface  to  the  metadata  files  that  assist  the  user  in  making  informed  and  rapid  queries  into 
the  contents  of  the  TACNEPH  data  sets.  The  application  is  open-ended  to  allow  for 
expansion  of  the  data  types  to  accommodate  future  needs  of  the  TACNEPH  database. 

3.2  Image  Processing  and  Display 

In  support  of  the  image  processing  and  cloud  analysis  tasks,  software  interfaces 
have  been  developed  to  tie  products  developed  for  these  tasks  into  the  database 
management  structure.  Image  processing  and  display  functions  include  programs  to 
selectively  query  the  database  and  load  satellite  imagery  and  analysis  products  onto  a 
display  device,  functions  to  manipulate  the  imagery  data,  and  a  program  to  assist  with 
manu^  analysis  of  satellite  imagery  data. 

An  image  display  utility  called  TLOAD  loads  user-specified  areas  of  an  AVHRR  or 
OLS  image  into  Adage  memory.  The  utility  takes  advantage  of  the  TDB  interface  modules 
which  were  designed  to  return  platform-specific  information  to  an  application  which  calls 
them.  In  this  case,  the  number  of  channels  which  are  associated  wiA  a  particular  satellite 
are  loaded  as  8-bit  gray  shade  images  into  specific  areas  of  Adage  refresh  memory,  in  an 
order  which  will  also  support  display  of  multispectral  color  composites.  Intermediate 
results  fi-om  a  cloud  analysis  algorithm  are  also  loaded  as  an  8-bit  synthetic  image  in  the 
overlay  planes  of  Adage  memory.  Each  of  the  cloud  and  background  tests  is  represented  as 
a  different  color  in  overlay  memory  and  can  be  viewed  separately  using  an  image  mani¬ 
pulation  and  data  retrieval  program  called  BIT_TOGGLE.  TLOAD  and  BIT_TOGGLE 
make  extensive  use  of  high  level  TDB  utilities  for  fast,  easy  access  to  data  and  analyses. 

Bit  togGLE  provides  an  interactive  utility  to  manipulate  and  display  satellite 
imagery  and  analysis  algorithm  results  quickly  and  easily.  It  was  used  extensively  during  the 
cloud  adgorithm  test  and  evaluation  phases  of  the  project  described  in  Section  5.3.  The  main 
program  feature  is  a  user  interface  that  combines  keyboard  and  mouse-driven  features  to 
obtain  information  about  the  imagery  on  a  pixel  level,  such  as  earth  location,  satellite  bright¬ 
ness  temperature  or  visible  count,  or  corresponding  surface  temperature  climatology  values. 
Bn'_TOGGLE  display  capabilities  include  rapid  toggling  through  different  channels  of 
display  memory  to  view  imagery  from  different  sensor  charmels,  full-color  multispectral 
display  techniques  to  distinguish  different  types  of  cloud  and  background  surfaces  (see 
dEntremont  et  al.,  1987),  and  display  of  individual  color-mapped  cloud  test  results  over 
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single  channel  and  multispectral  composites.  By  altering  the  selection  of  individual  planes, 
an  analyst  can  evaluate  the  algorithm  intermediate  results  for  a  particular  region  of  interest. 

Validation  of  algorithm  results  required  the  manual  analysis  of  a  large  set  of  satellite 
images  (see  Section  5.3.2).  An  interactive  program  called  TBLANK  was  developed  to 
assist  the  analyst  in  this  process.  Visualization  of  sensor  data  is  controlled  through  a  set  of 
fast  interactive  person/machine  functions  that  support  both  single  channel  gray  shade  and 
full  color  multispectral  display,  image  enhancement,  segmentation  and  thresholding. 

Actual  delineation  of  cloud  boundaries  is  accomplished  through  a  technique  known  as 
threshold  blanking.  Here  the  analyst  selects  a  region  of  interest,  displays  the  sensor  data  in 
whatever  format  dlows  for  optimd  interpretation,  and  interactively  raises  or  lowers  a  pixel 
intensity  threshold  to  separate  cloud-filled  from  clear  pixels.  The  threshold  can  be  either 
coarsely  or  finely  adjusted  until  it  accurately  delineates  the  cloud  boundary  as  determined 
by  the  analyst.  The  magnitude  of  the  threshold  level  is  viewed  as  a  user  selectable  color 
shading  of  the  image.  For  example,  if  the  analyst  chooses,  say,  a  green  shade  then  pixels 
with  intensity  levels  below  the  threshold  are  displayed  as  shades  of  green  while  pixels 
above  retain  their  original  color  shading.  Thus,  the  analyst  can  easily  see  where  the 
threshold  level  is  set  without  obscuring  any  features  with  intensity  levels  above  or  below 
that  level.  The  size  and  sensor  channel  selection  for  each  Region  of  Interest  (ROI)  are 
completely  at  the  control  of  the  analyst  and  can  be  changed  at  any  time  during  the  process. 
Normally  the  analyst  segments  an  image  many  times  using  different  thresholds  and  channel 
combinations  depending  on  the  diversity  of  the  cloud  and  background  conditions,  thus,  the 
process  tends  to  be  iterative.  Threshold  blanking  has  been  found  to  be  a  fast  and  highly 
accurate  method  for  transferring  an  analyst's  interpretation  into  a  digital  form. 

4 .  Clear-Scene  Background  Characterization 

The  TACNEPH  algorithm  generates  two  internal  databases  that  provide  statistical 
information  on  the  visible  and  infrared  characteristics  of  the  cloud-free  background.  They 
comprise  dear-scene  statistics  that  are  required  for  discrimination  of  cloud-free  from  cloud- 
contaminated  radiative  signatures  in  single  channel  cloud  tests  used  in  both  the  OLS  and 
AVHRR  cloud  algorithms.  Table  5  summarizes  the  required  internal  databases  used  to 
characterize  the  cloud-free  background.  The  data  must  be  stratified  by  satellite  and  time  of 
day  (i.e.,  separate  dear-scene  statistics  for  ascending  and  descending  orbits  of  each  satel¬ 
lite).  The  infrared  statistics  are  used  indirectly  to  adjust  the  surface  temperature  climatology 
values  in  order  to  predict  the  AVHRR  channel  4  and  OLS/T  brightness  temperatures  that 
would  be  measured  under  cloud-free  conditions.  The  visible  statistics  are  used  directly  for 
comparison  with  the  measured  visible  counts  (OLS)  or  albedo  (AVHRR).  During  the 
TACNEPH  development  program,  algorithm  testing  and  validation  was  performed  over 
numerous  ROI  that  were  boxes  that  ranged  from  a  few  hundred  to  approximately  KXK)  km 
on  a  side  (see  Section  5.3  for  a  description  of  the  ROI  regions).  Clear-scene  statistics  were 
maintained  separately  for  each  ROI  to  individually  characterize  the  natural  variability  of  the 
measured  satellite  data  under  cloud-free  conditions.  In  the  tactical  terminal  implementation, 
it  is  likely  that  a  single  set  of  statistics  will  be  sufficient  for  the  entire  tactical  area  unless 
there  is  a  well-defined  boundary  between  substantially  different  background  types,  such  as 
a  coastline.  In  that  case  it  would  be  necessary  to  generate  two  separate  statistical  databases, 
one  characterizing  the  land  surfaces  and  the  other  characterizing  water  surfaces.  This  type 
of  situation  can  be  readily  automated  through  use  of  the  TACNEPH  geography  database  to 
discriminate  the  two  classes.  This  was  in  fact  done  in  the  research  algorithin  whenever  a 
given  ROI  contained  a  significant  fraction  of  both  land  and  water  surfaces.  Further 
discussion  of  background  stratification  can  be  found  is  Section  5.3.1.  The  visible  and 
infrared  statistics  databases  are  described  in  Sections  4.1  and  4.2,  respectively. 
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Table  5.  Clear-scene  Internal  Database  Characteristics 


Type 

Resolution 

Projection 

Description 

clear-scene  IR 
statistics 

ROI 

history  of  LWIR-Tr.iim  differences 

clear-scene  visible 
statistics 

0.1° 

lat/lon 

clear-scene  albedo  normalized  by  BRDF 

4.1  Clear-Scene  Visible  Databases  for  OLS  and  AVHRR 

Reference  clear-scene  background  visible  channel  data  are  required  for  the  single 
channel  visible  or  near-IR  reflectance  tests  in  the  OLS  and  AVHRR  algorithms.  Visible/ 
near-IR  clear-scene  data  are  processed  in  a  manner  similar  to  the  RTNEPH  background 
brightness  database.  For  each  ROI  a  separate  database  of  clear-scene  albedo  is  maintained 
for  each  daytime  satellite  overpass  time.  The  data  are  maintained  over  a  regular  latitude  - 
longitude  grid  with  an  interval  of  0.1  degrees  and  are  updated  continuously  as  new  data  are 
received.  Clear-scene  information  is  produced  from  by-products  of  the  cloud  analysis 
algorithms.  As  satellite  data  are  received  and  analyzed  through  the  appropriate  nephanaly- 
sis  algorithm,  two  passes  through  the  algorithm  are  performed:  1)  cloud  detection,  and  2) 
cloud  clearing  (see  Section  5).  In  cloud  detection  mode,  algorithm  thresholds  are  set  to 
provide  an  optimal  analysis  with  no  preference  toward  over  or  under-analysis  of  cloud.  In 
cloud  clearing  mode,  cloud  thresholds  are  set  with  a  bias  toward  over-analysis  to  insure 
identification  and  removal  of  all  cloud-contaminated  pixels.  In  the  current  implementation 
of  the  algorithms,  the  clear  scene  information  obtained  from  cloud  clearing  is  used  to 
update  the  databases  for  the  next  satellite  pass  rather  than  the  current  analysis.  Some  small 
benefit  may  be  gained  by  updating  the  databases  prior  to  performing  the  cloud  detection 
part  of  the  current  analysis. 

For  each  0. 1  degree  grid  box,  all  clear-scene  pixel  values  are  accumulated 
separately  for  the  OLS-L  and  AVHRR  channel  2  and  the  mean  count  (OLS)  or  albedo 
(AVHRR)  value  is  calculated.  For  the  AVHRR,  the  mean  albedo  is  then  normalized  by  an 
anisotropic  reflectance  factor  (ARF)  to  adjust  for  the  effect  of  different  relative  solar  and 
satellite  angles  across  the  scene: 


A'i  =  Ai/ARF(V|/,e,<l),M)  (4-1) 

where  Aj  and  A'j  are  the  measured  and  normalized  albedo  for  channel  i=l,  2,  respectively, 
M  is  the  Geographic  Type  of  the  background  surface  (i.e.,  land  or  water),  and  \)/,  0,  and  (j) 
are  the  satellite  zenith,  solar  zenith,  and  sun-satellite  azimuth  angles,  respectively.  Evalua¬ 
tion  of  the  ARF  function  is  performed  through  look  up  tables  published  by  Taylor  and 
Stowe  (1984).  The  ARF  function  is  not  applied  to  the  OLS  data  because  on-board  satellite 
gain  controls  are  designed  to  perform  this  same  function.  Clear-scene  data  are  filtered 
through  a  60°  solar  zenith  angle  threshold  to  insure  only  daytime  data  for  which  ARF 
values  are  available  are  processed.  To  ensure  that  cloud  contaminated  data  does  not  corrupt 
the  database,  new  information  is  added  only  if  the  value  does  not  exceed  the  magnitude  of 
the  stored  value  by  more  than  0. 1  (new  data  can  be  darker  by  any  amount).  The  database  is 
updated  using  a  weighted  average  of  the  previous  value  and  the  new  normalized  albedo: 

V  =  0.25  V  +  0.75  V  (4-2) 
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where  V  is  the  stored  dear-scene  visible  count  or  normalized  albedo  and  V  the  newly 
measured  value.  The  gross  fidelity  of  this  database  can  be  established  easily  by  displaying 
the  data  as  an  image. 

4.2  Clear-Scene  Infrared  Databases  for  OLS  and  AVHRR 

The  AVHRR  and  OLS  cloud  algorithms  both  include  a  single  channel  infrared 
threshold  test  and  as  such  require  estimates  of  dear-column  satellite  brightness  tempera¬ 
tures  to  discriminate  cloud-free  from  cloud  contaminated  radiative  signatures.  In  the 
TACNEPH  program,  the  only  database  providing  any  information  related  to  dear-column 
brightness  temperatures  is  the  Surface  Temperature  Climatology  database  described  in 
Section  3.  The  required  predicted  dear-scene  satellite  brightness  temperatures  are  com¬ 
puted  using  a  dynamic  correction  to  the  climatology  database.  The  correction  accounts  for 
the  combined  effect  of  the  multiple  error  sources  that  can  occur  when  using  a  fixed 
climatology  of  shelter  temperatures  to  predict  a  satellite  derived  dear-scene  brightness 
temperature.  Of  particular  concern  are  shelter  temperatures  that  are  not  representative  of 
bandpass-weighted  satellite  infrared  brightness  temperatures,  differing  spatial  resolutions 
between  the  climatology  and  satellite  derived  data,  satellite  sensor  calibration  errors,  and  the 
effect  of  IR  atmospheric  attenuation.  Accurate  modeling  of  individual  errors,  let  alone  their 
combined  effect,  is  problematic  so  the  approach  selected  for  TACNEPH  uses  a  single 
correction  factor  to  account  for  all  error  sources  collectively. 

The  calculation  of  the  predicted  brighmess  temperature  that  would  be  observed  by  a 
satellite  from  the  cloud-free  terrestrial  background  at  a  given  location  and  time  is  performed 
as  part  of  the  cloud  analysis  algorithm;  however  an  internal  database  of  dear-scene  infrared 
statistics  must  first  be  generated.  The  initial  step  is  to  compile  a  ten-day  record  of  the 
deviation  of  the  climatology  temperature  (Tclim)  from  the  satellite  derived  temperature 
(Tsat)  for  locations  previously  classified  as  cloud-free.  This  record  is  used  to  characterize 
the  natural  variability  of  the  difference  between  the  two  temperature  values  when  no  cloud 
is  present  so  that  future  measurements  can  be  tested  to  see  if  they  fall  within  the  expected 
range  for  clear  conditions. 

Temperature  difference  information  is  maintained  as  an  internal  statistical  database 
summarizing  the  distribution  of  the  temperature  differences  stratified  by  location,  satellite, 
time  of  day,  and  surface  type:  land,  water,  or  desert.  Statistics  are  accumulated  over  the 
tactical  region  of  interest.  Thus,  a  separate  database  entry  is  required  for  each  of  the  ten 
days,  for  each  polar  satellite,  its  ascending  and  descending  orbits,  and  the  three  possible 
surface  types  identified  in  the  geographic  supporting  database. 

As  with  the  visible  data  described  above,  cloudy  pixels  are  removed  by  analyzing 
the  data  through  the  appropriate  cloud  algorithm  mn  in  cloud-clearing  mode.  Once  clouds 
have  been  identified  and  removed  from  further  processing  and  the  difference  between  the 
sensor  channel  brightness  temperature  and  the  corresponding  surface  climatology  tempera¬ 
ture  is  calculated  for  all  remaining  pixels  and  used  to  generate  a  frequency  distribution. 
Figure  2  illustrates  a  sample  temperature  difference  distribution  developed  from  89,763 
cloud-free  pixels  observed  during  a  NOAA  1 1  afternoon  ascending  pass  over  land  surfaces 
in  the  east  central  United  States. 

If  n  =  1, 2, ...,  Ni  is  the  number  of  clear  pixels  from  the  cloud  clearing  analysis  for 
some  day  i  =  1, 2, 10  then  for  each  pixel  n,  the  temperature  difference  (ATn)  is  defined 
as: 


ATn  =  Tsatn-Tclimn  , 


(4-3) 


20 


^Tinin 

Tsat  -Tclim 


Figure  2.  Example  histogram  of  comparison  between  satellite  brightness  temperature 
and  corresponding  climatology  temperature  value  for  89,763  clear  pixels 
obtained  from  an  AVHRR  scene  from  1947  UTC  on  6  June  1992.  Vertical 
lines  represent  2  standard  deviations  about  the  mean. 

where  Tsatn  is  the  OLS  or  AVHRR  brightness  temperature  of  a  pixel  n  and  Tclimn  is  a  time 
interpolated  climatology  temperature  vdue,  derived  from  the  Surface  Temperature 
Climatology  database,  that  corresponds  to  the  time  and  location  of  pixel  n.  Tclimn  is 
defined  by  first  locating  the  1/8^^  mesh  grid  box  closest  to  the  latitude  and  longitude  of 
pixel  n  (note  the  AFGWC  polar  grid  convention  uses  the  upper  left  comer  to  define  the 
location  of  a  grid  box,  see  Hoke  et  al.,  1981).  The  two-step  time  interpolation  is  used. 
First,  to  match  the  date  of  the  satellite  data,  and  second  to  match  the  time  of  day.  The  date 
interpolation  is  performed  to  avoid  discontinuities  in  the  dear-scene  statistics  over  month 
boundaries  (recall  that  the  three-hourly  climatology  data  are  updated  monthly).  The  first 
step  to  interpolate  the  climatology  data  to  the  valid  satellite  date  is  performed  by  assuming 
the  climatologies  are  valid  on  the  15th  of  each  month.  To  interpolate  to  time  of  day,  the 
two  climatology  temperature  database  entries  with  valid  times  that  bracket  the  time  of  the 
satellite  observation  are  located  and  the  respective  shelter  temperature  values  at  the  specified 
1/8“^  mesh  grid  point  are  linearly  interpolated  to  the  time  of  the  satellite  observation: 

Xciim„  -  -  -  (tsat~tl)  +  Tclim,,  (4-4) 

where  tsat  is  the  satellite  observation  time,  ti  and  t2  are  the  valid  times  of  the  bracketing 
climatology  temperature  database  entries  (i.e.,  ti  <  tsat  <  t2),  and  Tclimtl  and  Tclimt2  are 
the  resp^tive  climatology  temperature  values  for  the  specified  grid  box  valid  at  times  ti  and 
t2.  "Hie  interpolation  process  is  not  critical  to  the  dear-scene  temperature  estimation  except 
that  it  tends  to  eliminate  artificial  boundaries  in  the  analysis  when  the  sensor  data  passes 
from  one  climatological  time  regime  to  another  (e.g.,  from  one  synoptic  time  to  another  or 
over  the  end  of  one  month  into  die  next). 
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The  general  Gaussian  shape  of  the  distribution  in  Figure  2  is  typical,  therefore  it 
was  decided  that  the  range  of  ATf  values  found  for  each  day  i  could  be  represented  using 
the  limits  defined  by  two  standard  deviations  taken  about  the  mean  of  the  temperature 
difference  distribution  (labeled  ATmjn  and  ATmax  in  Figure  2).  Thus,  the  IR  Temperature 
Statistics  database  contains  an  historical  record  of  the  AT  values  corresponding  to  the  2a 
limits  for  each  location,  satellite,  orbit,  and  background  combination.  For  day  i  the  mean 
difference: 


J_ 

N 


i  n=l 


(4-5) 


and  the  standard  deviation: 
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(4-6) 


are  computed  and  used  to  define  the  2a  limits  (ATminj  and  ATmaxj)  that  represent  the 
extremes  of  the  AT  distribution: 


ATmin.  =AT-2a.  (4-7) 

and 

ATmax.  =  AT -I- 2a..  (4-8) 

Once  the  IR-Climatology  Temperature  Statistics  have  been  accumulated  for  the 
previous  10  days  they  are  used  by  the  OLS  and  AVHRR  cloud  detection  algorithms  to  help 
predict  the  brightness  temperature  (Tpred)  that  would  be  measured  from  the  satellite  in  the 
absence  of  cloud  for  the  current  time,  location  and  background  type.  The  procedure  is 
applied  to  the  thermal  infrared  channel  data  from  each  sensor  (AVHRR  charmel  4  and 
OLS-T).  First,  the  satellite  data  being  analyzed  are  segmented  into  a  series  of  small 
analysis  regions.  During  TACNEPH  it  was  set  empirically  at  16  x  16  pixels.  However, 
these  numbers  should  be  considered  a  minimum  size  since  testing  has  indicated  that,  in 
practice,  it  can  be  increased  if  necessary  to  improve  computation^  efficiency.  The  critical 
fector  in  determining  the  size  of  the  andysis  region  is  whether  the  spatial  variability  of  the 
background  temperature  resolved  by  the  satellite  data  is  captured  in  the  climatology 
temperature  database  (i.e.,  is  the  magnitude  of  the  temperature  variation  over  the  analysis 
region  approximately  the  same  in  the  climatology  temperature  database  as  in  the  satellite  IR 
channel  data). 

After  the  data  are  segmented  the  next  step  is  to  compute  a  separate  correction  factor 
for  each  analysis  region.  Recall  that  the  correction  will  be  added  to  the  1/8^^  mesh  clima¬ 
tology  temperatures  to  predict  the  local  (i.e.,  16  x  16)  dear-scene  brightness  temperatures. 
One  pixel  from  the  analysis  region  that  is  considered  most  likely  to  be  cloud-free  is  selected 
and  used  to  establish  a  reference  satellite-climatology  temperature  difference  value  (ATref) 
for  the  entire  local  region.  To  minimize  the  likelihood  of  cloud  contamination  the  reference 
pixel  is  taken  as  the  warmest  pixel  in  the  analysis  region.  To  avoid  using  a  warm  anomaly 
to  establish  the  reference  value,  the  warmest  1%  of  all  pixels  in  the  analysis  region  are  first 
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removed  before  the  reference  pixel  is  selected  (e.g.,  for  a  16  x  16  region,  pixels  with  the 
two  highest  brightness  temperatures  are  excluded^  Thus,  ATref  is  defined  as: 

ATref=Tref-Tclim  (4-9) 


where  Tref  is  the  brightness  temperature  of  the  reference  pixel  and  Tclim  is  the  time 
interpolated  climatology  temperature  corresponding  to  the  time  and  location  of  the  reference 
pixel  (calculated  in  the  same  way  as  Tclimn  in  Eq.  4-4). 

If  the  reference  pixel  can  be  established  to  be  cloud-free  then  ATref  is  used  as  the 
climatology  temperature  correction  for  its  respective  analysis  region.  However,  the  critical 
step  affecting  cloud  analysis  accuracy  is  testing  of  the  reference  pixel  for  possible  cloud 
contamination.  If  the  reference  pixel  does  contain  cloud  then  the  predicted  dear-scene 
brightness  temperature,  Tpred,  will  be  representative  of  the  cloud  brightness  temperature 
and  not  that  of  the  terrestrial  background.  Testing  of  the  reference  pixel  is  accomplished  by 
comparing  the  magnitude  of  ATref  against  the  range  of  expected  dear-scene  temperature 
differences  established  from  the  ten-day  IR-Climatology  Temperature  Statistics.  If  ATref 
falls  within  the  expected  range,  then  the  reference  pixel  is  assumed  to  be  cloud-free, 
otherwise  it  is  assumed  to  be  cloud  contaminated  and  a  default  value  is  used  for  ATref.  To 
establish  the  expected  range  of  dear-scene  values  a  time  and  frequency-weighted  average  of 
the  historical  dear-scene  AT  limits  is  used: 
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where  a  and  b  are  empirically  defined  coefficients  for  the  temporal  and  frequency  average 
terms,  respectively:  the  sum  of  a  -h  b  must  equal  1.0,  currently  a  =  0.9  and  b  =  0.1.  The 
time  weighting  factor  q  is  defined  to  give  greatest  weight  to  the  most  recent  day  and 
decreases  as  the  dear-scene  data  age.  To  avoid  the  use  of  anomalous  data  in  the  time- 
frequency  averaging  process,  a  minimum  sample  size  is  required.  For  data  from  a  given 
day  to  be  included  in  the  ten-day  average,  the  number  of  dear-scene  data  points  in  Ae 
distribution,  Ni,  must  exceed  5000.  Any  days  for  which  Nj  is  less  than  5000  are  excluded 
from  the  averaging  process  and  data  from  the  next  oldest  day  are  added  to  the  series  to 
maintain  the  ten-day  total.  The  value  of  q  is  assigned  to  1  for  the  oldest  day  in  the  series 
and  increases  in  value  by  the  difference  in  Julian  date  from  the  date  of  the  oldest  day.  For 
example,  if  the  Julian  date  for  the  first  day  in  the  series  (i.e.,  i=l,  ti=l)  were,  say,  140, 
and  two  subsequent  days  had  sample  sizes,  Nj,  of  less  than  5000  then  the  Julian  date  of  the 
tenth  day  in  the  series  would  be  152  and  the  time  weight,  tio,  would  be  12  (152-140). 

Thus  to  calculate  a  predicted  brightness  temperamre  corresponding  to  any  pixel  n 
within  the  analysis  region  ATref  is  first  tested  for  cloud  contamination.  If: 
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ATmin  ^  ATref  ^  AXmax 


(4-12) 


then  ATref  is  assumed  to  be  cloud-free  and  is  added  to  the  time  interpolated  climatology 
temperature  corresponding  to  that  pixel  as  the  correction  factor: 


Tpredjj  —  Tclim^  +  ATref  ,  (4-13) 

Otherwise  ATref  is  assumed  to  be  cloud  contaminated  and  a  default  correction  based  on  the 
mean  of  the  time-frequency  weighted  average  AT  limits  is  used  to  calculate  the  predicted 
dear-scene  brightness  temperature: 


Tpred^  —  Tclim^  +  “  ^ATmin  +  AT  max)  . 


(4-14) 


As  stated  above,  the  predicted  dear-scene  brightness  temperature  (Tpred)  is  cal¬ 
culated  in  exactiy  the  same  way  by  both  the  OLS  and  AVHRR  cloud  analysis  algorithms. 
However,  once  established  it  is  used  differently  by  each  algorithm  in  the  cloud  detection 
process  (see  Sections  5.1  and  5.2  for  descriptions  of  how  the  predicted  temperature  is  used 
in  the  OLS  and  AVHRR  algorithms,  respectively). 


4.3  SSM/I  Surface  Temperature 

Microwave  data  from  the  SSM/I  represent  another  potential  source,  in  addition  to 
the  Surface  Temperature  Climatology  database,  for  information  on  dear-scene  brightness 
temperature.  An  all  weather  multiple  regression  algorithm  was  developed  at  AFGWC  to 
use  SSM/I  seven  channel  radiometer  data  to  estimate  OLS  IR  brightness  temperature 
measurements  that  would  be  observed  from  the  terrestrial  surface.  Since  many  clouds  are 
transparent  at  the  SSM/I  frequencies,  this  technique  has  to  potential  to  provide  useful 
information  on  the  dear-scene  OLS  brightness  temperature  under  both  clear  and  cloudy 
conditions.  This  would  be  potentially  very  useful  to  the  TACNEPH  application  since,  as 
described  above,  accurate  information  on  the  cloud-free  environment  has  a  first-order 
impact  on  cloud  analysis  accuracy. 

The  SSM/I  algorithm  provided  by  AFGWC  is  a  two-step  linear  regression 
algorithm.  Both  steps  use  a  series  of  linear  coefficients  applied  to  the  seven  SSM/I 
channels  to  retrieve  the  desired  parameters.  Since  the  measured  radiance  at  each  channel  is 
strongly  affected  by  the  background  surface  (largely  due  to  variations  in  surface  moisture), 
the  first  step  characterizes  the  background  surface  type  into  one  of  30  categories.  Ideally, 
for  each  category  a  separate  set  of  regression  coefficients  would  be  available  for  the  second 
step  which  is  to  retrieve  the  dear-scene  IR  brightness  temperature. 

Unfortunately,  the  source  code  and  regression  coefficients  provided  for  the 
TACNEPH  program  had  been  developed  for  die  SSM/I  instrument  on  the  DMSP  F8 
satellite  and  were  incomplete.  Of  the  30  possible  background  surface  categories, 
coefficients  were  available  to  classify  and  process  data  over  only  five.  Also,  the  only 
DMSP  data  available  to  TACNEPH  were  obtained  from  the  FIO  and  FI  1  satellites  via  the 
PL  ground  stations.  No  F8  data  could  be  received  because  the  appropriate  decryption 
codes  were  not  available.  However,  the  F8  coefficients  were  applied  to  data  from  FI  1  on 
an  experimental  basis  to  determine  if  they  were  useful  for  estimating  OLS  dear-dear-scene 
temperatures.  Results  for  both  the  surface  type  classification  and  temperature  retrieval 
portions  of  the  algorithm  were  poor.  The  surface  classification  could  not  even  accurately 
discriminate  land  from  water  backgrounds  and  temperature  values  frequendy  differed  from 
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coincident  OLS  dear-scene  brightness  temperatures  by  greater  than  30  K.  Conversations 
with  personnel  at  AFGWC  confirmed  that  Ae  disappointing  performance  of  the  algorithm 
was  probably  due  to  the  use  of  F8  coefficients  with  FI  1  data. 

During  the  TACNEPH  program  period  of  performance,  work  continued  at 
AFGWC  to  implement  a  capability  to  develop  regression  coefficients  for  new  satellites  and 
to  monitor  performance  of  Ae  temperature  retrieval  algorithm  over  time.  While  this  effort 
was  apparently  successful,  no  additional  coefficients  were  received  in  time  to  be  included  in 
the  TACNEPH  effort.  Because  of  this,  the  technique  could  not  be  directly  evaluated  for 
use  in  the  TACNEPH  algorithms.  However,  based  on  work  that  was  accomplished,  some 
observations  and  recommendations  related  to  the  TACNEPH  program  can  be  provided. 

4.3.1  SSM/I  Antenna  Pattern  Correction 

SSM/I  sensor  measurements  are  calibrated  to  produce  antenna  temperatures.  These 
must  be  corrected  for  interactions  between  antenna  characteristics  and  the  upwelling 
radiation  (i.e.,  antenna  pattern  corrections,  APC)  to  produce  physical  brightness 
temperatures  required  by  the  EDR  algorithms,  hi  the  course  of  early  investigations  on  this 
task  it  was  discovered  that  at  least  two  diflFerent  approaches  are  available  to  perform  antenna 
pattern  corrections  to  the  SSM/I  (Wentz,  1988  and  Hollinger  et  al.,  1987).  The  principal 
difference  between  the  two  is  that  the  technique,  in  addition  to  correcting  for  feed 
horn  characteristics  and  cross-polarization  coupling,  applies  a  second  order  side  lobe 
correction  that  is  absent  in  the  Wentz  approach.  Additional  information  was  requested  and 
received  from  AFGWC  on  the  APC  technique  in  use  there.  The  AFGWC  technique  was  a 
variation  on  the  Wentz  technique  through  a  somewhat  different  implementation  of  the 
cross-polarization  coupling  correction.  However,  APC  adjusted  brightness  temperatures 
obtained  from  the  two  algorithms  showed  good  agreement  over  a  range  of  temperatures. 

Indef^ndent  experiments  using  the  NRL  two  level  technique  also  showed  good 
agreement  with  both  the  Wentz  and  AFGWC  correction  algorithms  for  cases  where  a 
uniform  array  of  antenna  temperatures  were  used  to  exercise  the  algorithms.  This  confirms 
that  the  three  techniques  are  comparable  for  feed  hom  and  cross-polarization  coupling 
corrections  since  the  NRL  second  order  corrections  account  for  contributions  from  side 
lobes  and  should  not  cause  any  change  in  temperature  if  the  side  lobe  energies  are  the  same 
as  the  main  beam.  Figure  3a  compares  the  N^  and  AFGWC  level  1  corrections  to  the  37 
GHz  horizontal  polarization  channel  for  approximately  125  consecutive  scan  lines  taken 
from  a  single  orbit.  For  the  level  2  corrections  to  be  non-zero  some  gradient  has  to  exist  in 
the  array  of  antenna  temperatures.  Anecdotally,  it  was  learned  that  there  is  generally  little 
variation  over  in-band  antenna  temperatures  across  a  single  scan,  but  along  scan  there  can 
be  a  wide  range  of  differences.  Figure  3b  illustrates  the  magnitude  of  the  level  2 
corrections  for  the  same  set  of  scan  lines  contained  in  Figure  3a. 

It  was  not  possible  to  evaluate  the  impact  of  the  NRL  level  2  corrections  on 
retrieved  IR  brightness  temperatures  due  to  our  inability  to  successfully  run  the  SSM/I 
algorithm  using  the  outdated  coefficients.  However,  the  information  in  Figure  3  is 
suggestive  of  some  general  observations.  NRL  level  2  corrections  do  not  appear  to 
significantly  affect  Ae  magnitude  of  the  derived  brightness  temperature  under  most 
conditions.  Although  for  some  situations,  presumably  when  there  is  a  strong  along  track 
gradient,  the  level  2  corrections  become  significant  and  can  exceed  the  magnitude  of  the 
level  1  corrections. 
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Figure  3a.  Comparison  of  the  NRL  and  AFGWC  level  1  corrections  to  the  37  GHz 
horizontal  polarization  channel  for  approximately  125  consecutive  scan 
lines  taken  from  a  single  orbit. 


Figure  3b.  Magnitude  of  the  level  2  cor^ctions  for  the  same  set  of  scan  lines 
contained  in  Figure  3a. 


4.3.2  Application  of  SSM/I  Derived  Surface  Temperature 

Estimates  of  OLS  dear-scene  IR  brightness  temperatures  retrieved  from  the  SSM/I 
are  potentially  useful  in  TACNEPH  in  some  situations.  As  expected,  the  existing  Surface 
Temperature  Climatology  data  have  been  found  to  vary  significantly  from  the  satellite 
observed  IR  temperatures  over  dear-scene  conditions.  While  the  dynamic  correction 
described  in  Section  4.2  that  is  used  to  predict  the  dear-scene  IR  temperature  ^vorks  well 
over  most  situations,  there  are  certain  conditions  where  improved  first-guess  estimates 
would  be  particularly  beneficial.  One  class  of  conditions  where  this  is  true  is  when  there 
has  been  a  rapid  change  in  the  thermal  characteristics  of  the  terrestrial  background, 
particularly  since  the  last  satellite  overpass.  Examples  include  a  snow  fall  or  snow  melt  or 
even,  under  extreme  conditions,  a  frontal  passage.  In  these  cases  the  ten  day  historical 
record  used  to  characterize  the  expected  range  of  variation  between  satellite  and 
climatological  temperatures  may  not  be  applicable.  Generally  the  algorithm  requires  a  few 
days  to  react  to  these  t^s  of  rapid  change,  during  which  time  algorithm  performance 
degrades.  This  is  particularly  important  for  OLS  processing  since  that  algorithm  is  most 
dependent  on  the  single  channel  IR  test. 

Where  they  overlap  the  OLS,  SSM/I  data  have  the  obvious  advantage  over 
dinaatology  of  temporal  consistency.  As  such  they  would  be  expected  to  resolve  the  types 
of  situations  discussed  above  that  currently  impact  OLS  algorithm  performance.  The 
recommended  approach  to  using  SSM/I  derived  temperatures  is  identical  to  that  described 
iri  Section  4.2  for  using  climatological  temperatures.  A  record  of  the  comparative 
differences  between  OLS  and  SSM/I  temperatures  would  be  generated  for  locations 
classified  as  cloud-free.  From  this  an  expected  range  of  allowable  differences  would  be 
generated  for  each  pass  and  compared  to  the  actual  measured  differences.  This  would  be 
necessary  to  account  for  variations  in  the  SSM/I  retrievals  caused  by  different  terrestrial 
backgrounds  and  certain  cloud  types  (e.g.,  precipitating  cloud).  If,  as  expected,  it  turns 
out  that  the  SSM/I  derived  temperatures  are  a  better  first-guess  to  the  OLS  brightness 
temperatures  than  climatology,  the  result  would  be  a  smaller  range  of  2a  values  in 
equations  4-7  and  4-8,  which  in  turn  should  translate  to  an  improved  cloud  analysis. 


5.  Nephanalysis  Algorithms 

This  section  provides  a  description  of  the  TACNEPH  cloud  detection  algorithms 
which  are  designed  to  analyze  DMSP/OLS  and  NOAA/AVHRR  sensor  data.  It  is  intended 
as  the  primary  resource  for  further  implementations  of  the  algorithm  in  operational  environ¬ 
ments.  Other  secondary  resources  that  may  also  be  exploited  are  the  separate  software 
design  report  and  source  code  listings  that  describe  the  research  grade  code  produced  to 
support  the  algorithm  development  effort.  However,  it  should  be  kept  in  mind  that  the 
research  grade  code  was  designed  specifically  to  meet  the  requirements  of  the  Research  and 
Development  program  and  not  those  of  an  operational  tactic^  terminal  program.  As  such, 
any  use  of  the  research  grade  code  in  a  Technology  Transfer  (T2)  effort  should  be  for 
guidance  on  sizing  and  as  an  example  of  one  way  to  implement  the  algorithms.  Primary 
reliance  should  be  on  the  algorithm  descriptions  contained  in  this  section  of  the  report.  It  is 
designed  to  provide  a  step  by  step  description  of  the  algorithm  process  and  is  not  tied  to 
any  hardware  platform  or  software  environment;  however,  occasional  references  to  data  file 
formats  are  included  to  illustrate  how  data  specific  aspects  of  the  program  could  be  imple¬ 
mented.  The  research  grade  code  was  necessarily  developed  to  operate  in  a  VAX/VMS 
environment  and  contains  many  features  specifically  designed  to  assist  the  research  scientist 
in  evaluating  the  algorithm  performance  and  scientific  validity  of  the  approach.  Any 
features  of  this  type  that  have  potential  use  in  the  operational  implementation  will  be 
identified  in  the  text. 
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TACNEPH  cloud  analysis  algorithms  were  designed  to  address  the  graceful 
degradation  requirement  imposed  on  this  program.  The  DMSP  and  NOAA  algorithms  are 
completely  separate  approaches,  each  designed  to  exploit  the  unique  information  content  of 
the  respective  sensors.  The  algorithms  consist  of  a  series  of  tests  that  individually  analyze 
multiple  cloud  signatures  in  data  streams.  To  produce  a  final  cloud  analysis,  the  combined 
results  of  all  tests  are  analyzed.  Graceful  degradation  is  achieved  through  selective  evalua¬ 
tion  of  individual  cloud  tests  based  on  available  data.  For  example,  the  OLS  algorithm  in 
its  optimal  configuration  uses  both  visible  and  infrared  channel  data.  However,  at  night,  in 
sun-glint  conditions,  or  when  visible  channel  data  are  noisey  the  algorithm  will  operate  on 
single  channel  data.  The  fundamental  goal  during  the  TACNEPH  development  effort  was 
to  design  algorithms  that  will  produce  an  optimal  cloud  analysis  from  whatever  data 
sources  are  available. 

Rehance  on  external  data  sources  by  both  algorithms  is  strictly  limited  to  the  data 
sets  identified  in  Section  2.  These  particular  data  sets  were  identified  because  they  are 
assumed  to  be  available  in  a  tactical  terminal.  The  only  dynamic  data  requirements  are  for 
satellite  sensor  and  ephemeris  data,  all  supporting  data  are  static  statistic^  or  fixed  para¬ 
meters.  All  other  required  data  are  generated  internally  by  the  algorithms  (see  Section  4). 

For  the  purposes  of  algorithm  development,  cloud  analysis  results  are  provided  on 
a  pixel-by-pixel  basis.  Since  no  final  output  format  was  specified  it  was  felt  that  a  pixel 
analysis  would  be  the  most  flexible  in  terms  of  supporting  whatever  application  specific 
output  format  is  required  operationally.  From  this  format,  it  is  straightforward  to  compute 
the  required  cloud  amount  and  altitude  parameters  over  any  regular  gird  or  pixel  spacing. 

5 . 1  DMSP/OLS  Cloud  Analysis  Algorithm  Description 

The  TACNEPH  OLS  cloud  detection  algorithm  reUes  on  three  distinct  cloud  tests 
which  are  applied  separately  depending  upon  availability  of  the  required  image  data.  The 
tests  are  all  statistical  threshold  types  of  algorithms  and  consist  of  single  channel  visible  and 
infrared  tests  plus  a  two  channel  bispectral  test.  Results  of  each  test  are  stored  internally  as 
an  intermediate  database.  For  the  research  grade  implementation,  diagnostic  results  were 
also  stored  for  each  analysis  region  (defined  as  16  x  16  pixels)  where  the  size  of  the 
analysis  region  was  determined  at  run-time.  This  information  was  extremely  useful  as  a 
diagnostic  for  algorithm  performance  and  the  fine-tuning  of  parameters  which  affect  cloud 
detection.  If  this  information  should  be  determined  to  be  useful  in  the  operational 
implementation,  it  could  be  easily  produced  in  a  format  conducive  to  a  tactical  terminal  file 
system. 

The  multiple  cloud  tests  used  in  the  TACNEPH  OLS  algorithms  are  designed  to 
satisfy  the  graceful  degradation  requirement  of  the  TACNEPH  program.  Each  test  exploits 
cloud  information  based  upon  the  presence  of  infrared  and  visible  data;  however,  in  the 
event  of  lost  or  unusable  data,  the  algorithm  will  continue  to  produce  a  cloud  analysis  that 
optimally  utilizes  whatever  data  are  available. 

5.1.1  Threshold  Calculation 

The  TACNEPH  OLS  algorithms  require  two  dynamic  databases  to  predict  the 
infrared  and  visible  characteristics  of  the  dear-scene  background  (see  Section  4).  Clear- 
scene  statistics  are  required  for  discrimination  of  cloud-free  from  cloud-contaminated 
radiative  signatures  in  single  and  dual  channel  cloud  tests.  The  infrared  statistics  are  used 
indirectly,  to  adjust  surface  temperature  climatology  values  to  accurately  predict  infrared 
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brightness  temperatures  for  cloud-free  conditions.  The  visible  statistics  are  used  directly, 
for  comparison  with  the  measured  visible  brightness. 

The  threshold  approach  utilized  by  all  OLS  cloud  tests  was  selected  because  it 
allows  for  multiple  uncertainties  in  the  sensor  measurements,  including  sensor  calibration, 
dear-scene  characteristics,  and  atmospheric  transmission,  to  be  accounted  for  with  a  single 
value.  Recall  from  Section  4  that  an  empirically  derived  dynamic  correction  factor  is  used 
to  account  for  all  sources  of  error  collectively  without  the  need  to  understand  and  quantify 
the  individual  contributions.  The  magnitude  of  the  cloud  thresholds  is  then  dictated  by  the 
remaining  uncertainty  in  the  corrected  temperatures.  The  performance  of  the  algorithm  is 
directly  impacted  by  the  ability  to  accurately  characterize  cloud-free  backgrounds.  This  is 
achieved  through  the  identification  or  characterization  of  the  following: 

•  land/water/desert  boundaries, 

•  dear-scene  brightness  temperature,  and 

•  dear-scene  reflectance. 

Both  single  channel  and  bispectral  OLS  cloud  analysis  algorithms  use  dual 
threshold  cutoff  values  to  classify  clear,  cloud-filled,  and  partially  cloud-filled  pixels  within 
an  analysis  box.  The  method  employed  to  select  these  threshold  values  is  dependent  on  the 
sensor  ^ta  channel  being  analyzed,  infrared  or  visible. 

Infrared  thresholds  are  based  on  an  estimate  of  the  dear-scene  brightness 
temperature  derived  from  a  dynamic  correction  to  surface  skin  temperature  estimates 
obtained  from  a  surface  climatology  database  described  in  Section  4.2.  Once  a  predicted 
dear-scene  brightness  temperature  is  established,  threshold  values  are  computed  from 
statistical  estimates  of  the  expected  natural  variability  of  the  data. 

Clear-scene  infrared  statistics  information  is  used  both  to  establish  whether  a  given 
reference  pixel  is  cloud-contaminated  and  to  calculate  the  magnitude  of  the  clear  and  cloud 
threshold  values.  Thresholds  are  used  to  account  for  the  uncertainty  in  the  predicted  clear- 
scene  brighmess  temperature  calculation.  The  cloud  threshold  is  defined  by  the  equation: 

ATcid  =  I  ATmax  -  ATmin  I  *  OCdd  ,  (5-1) 

and  the  clear  threshold  is  defined  by: 

ATcir  =  I  ATmax  -  ATmin  I  *  OCclr »  (5-2) 

where  ATmax  and  ATmin  are  computed  using  Eq.  4-10  and  4-11  from  the  IR-Climatology 
Statistics  internal  database.  They  are  used  to  represent  the  natural  variability  of  the 
difference  between  the  satellite  observed  and  predicted  temperatures  for  the  cloud-free 
background.  The  values  of  Odd  and  ocdr  are  empirically  derived  coefficients  currently 
defined  as  1.5  and  0.75,  respectively. 

Once  the  thresholds  are  calculated  then  cloud  and  clear  cutoff  values  used  in  the 
analysis  algorithms  (see  following  sections)  are  calculated  by  subtracting  the  threshold 
values  from  the  predicted  clear-scene  brightness  temperature.  Thus,  the  cloud  cutoff  is 
defined  as: 


Tcld  =  Tpred  -  ATdd  ,  (5-3) 

and  the  clear  cutoff  value  as: 
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Tclr  =  Tpred-ATclr  •  (5-4) 

where  Tpred  is  calculated  as  in  Equations  4-13  and  4-14.  Calculation  and  application  of  the 
cutoff  values  is  illustrated  graphically  in  Figure  4. 


Frequency 

of 
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Figure  4.  IR  single  channel  test  dual  threshold  classification  approach. 


The  procedure  for  establishing  cutoff  values  for  the  visible  channel  follow  an 
approach  similar  to  that  for  the  infrared  channel.  Fundamental  differences  are  that  cloud 
and  clear  threshold  cutoff  values  are  calculated  individually  for  each  pixel  rather  than  using 
one  pair  of  values  over  a  whole  analysis  box,  and  the  technique  used  to  establish  cutoff 
values  varies  depending  on  the  geographic  type  of  the  scene.  Over  land  areas,  where 
surface  reflectance  is  expected  to  change  widi  time  and  geographic  region,  cutoff  values  are 
set  with  the  use  of  the  dynamically  maintained  Visible  Background  Count  internal  database 
described  in  Section  4. 1 .  Data  from  the  Visible  Background  database  are  used  in  a  way 
analogous  to  the  way  dear-scene  brightness  temperature  data  are  used  in  the  infrared 
technique.  However,  since  the  background  data  are  generated  directly  from  satellite 
observed  radiances  no  correction  factors  need  to  be  applied. 

If  the  pixel  being  analyzed  is  located  over  land,  then  the  method  for  determining  the 
cloud  cutoff  threshold  value  (Rcld)  is  defined  by  the  equation: 


^Id  ~  Rsfc  *  Pcld  >  (5-5) 

where  Rsfc  is  the  brightness  count  from  the  Visible  Background  database  (Section  4. 1)  that 
corresponds  to  the  satellite  pixel  and  pdd  is  an  empirically  derived  coefficient  used  to 
account  for  uncertainty  in  the  background  database.  Note  that  the  uncertainty  coefficient  is 
multiplied  by  the  background  value  rather  than  added  as  in  the  IR  cutoff  calculation  (Eq. 
5-3).  This  is  to  account  for  increasing  uncertainty  as  the  value  (i.e.,  brightness)  of  the 
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background  increases.  Similarly,  the  method  for  determining  the  clear  cutoff  value  (Rcir)  is 
defined  by  the  equation: 

Rclr  =  Rsfc  *  Pclr  »  (5-6) 

where  pdr  is  a  second  empirically  derived  coefficient.  The  values  of  pdr  and  pdd  are  0.4 
and  0.1,  respectively.  Once  the  visible  and  infrared  cutoff  values  are  established  they  are 
used  in  the  bispectral  algorithm  to  classify  cloud-filled,  cloud-free  or  partially  cloud-filled 
pixels.  Since  diere  is  not  a  one-to-one  correspondence  between  the  latitude/longitude 
mapping  of  the  background  database  (i.e.,  0.1°)  and  the  satellite  data,  new  clear  and  cloud 
cutoff  values  are  calculated  for  each  pixel. 

Over  water,  variations  in  surface  reflectance  are  considered  negligible  compared  to 
land.  These  cutoff  values  are  fixed  at  Rcid  and  Ret  are  fixed.  These  cutoff  values  are 
Reld“^^  and  Relr“37 . 

5.1.2  Single  Channel  Test 

The  DMSP  visible  and  infi*ared  single  channel  cloud  analysis  algorithm  is  applied 
when  data  from  only  one  OLS  channel  are  available.  Obviously,  the  most  common 
occumence  is  missing  visible  data;  however,  the  algorithm  can  be  applied  equally  to  either 
the  visible  or  infrared  channel  as  the  situation  demands.  If  both  visiWe  and  infrared  data 
are  available,  then  the  bispectral  test  described  below  is  applied.  The  single  channel 
algorithms  utilize  the  dual  threshold  approach  as  illustrated  in  Figure  4.  Separate  cutoff 
thresholds  are  defined  to  segregate  pixels  classified  as  partially  cloud-filled  from  those  that 
are  completely  cloud-filled  or  completely  cloud-free.  Cloud  analysis  accuracy  is  dependent 
on  the  accurate  prediction  of  the  dear-scene  brightness  temperature  used  to  define  the  clear 
and  cloudy  cutoff  values  as  described  above  in  Section  5.1.1. 

The  single  channel  algorithm  consists  of  two  tests.  First,  a  test  is  performed  to 
determine  if  the  brightness  temperature  of  the  IR  channel  is  less  than  the  cloud  cutoff  or  the 
visible  count  is  greater  than  the  cloud  cutoff: 

•Tir  <  Tcid  or 

*  Rvis  >  ^Id 

where  Tjr  is  the  OLS  infrared  brightness  temperature,  Tdd  is  the  IR  cloud  cutoff  value 
defined  by  Eq.  5-3,  Ryis  is  the  OLS  visible  count,  and  Rdd  is  the  visible  cloud  cutoff  value 
defined  by  Eq.  5-5.  For  the  IR  algorithm,  a  pixel  is  classified  as  cloudy  if  the  OLS 
brightness  temperature  is  less  than  the  cutoff  value.  Similarly,  the  visible  algorithm 
classifies  a  pixel  as  cloudy  if  the  visible  count  is  greater  than  the  cutoff  value. 

If  the  sensor  data  do  not  meet  the  cloudy  criteria  then  a  second  test  is  performed  to 
determine  if  the  pixel  is  completely  cloud-free.  This  is  done  by  testing  the  sensor  against 
the  clear  cutoff  values: 

*  Tir  >  Tcir  or 

*  Rvis  <  ^Ir 

where  Tdr  is  the  infi'ared  cloud-free  cutoff  defined  by  Eq.  5-4  and  Rdr  is  the  visible  cloud- 
free  cutoff  defined  by  Eq.  5-6.  If  these  tests  evaluate  as  true  then  the  pixel  is  classified  as 
cloud-free.  If  both  of  the  cloud  and  clear  tests  evaluate  as  false: 
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•  Tcid  <  Tir  <  Tcir  or 

*  Rcld  ^  Rvis  2:  Rcir 


then  the  pixel  is  classified  as  partially  cloud-filled  (i.e.,  the  FOV  of  the  sensor  contains  both 
cloud  and  clear). 

Once  pixels  have  been  classified  using  the  single  channel  algorithm,  fractional  cloud 
amount  can  be  calculated  over  some  spatial  region.  As  discussed  above,  the  size  and 
characteristics  of  this  region  are  assumed  to  be  application  dependent  since  no  specific 
requirements  were  imposed  on  TACNEPH.  To  compute  fractional  cloud  amount  over 
some  arbitrarily  defined  region  wherein  pixels  have  been  classified  as  either  clear,  partially 
cloud-filled,  or  cloudy,  the  contributions  from  each  class  need  to  be  quantified  as  follows: 


Ac  = 

'  N 


1 


N.u,+I 


(l«,  -  U) 

(^cld  ^clr) 


(5-7) 


where  Ac  is  the  effective  cloud  cover  (O  <  A^  <  l),  Ntot  is  the  total  number  of  pixels  in  the 
analysis  region,  Ndd  is  the  number  of  pixels  that  meet  the  completely  cloudy  criteria  (i.e., 
exceed  the  cloud  cutoff  value),  IjRj  is  the  measured  scene  radiance  of  pixels  classified  as 
partially  cloudy,  Icld  is  a  representative  cloud  radiance,  and  Iclr  a  representative  dear-scene 
radiance.  The  Ndd  term  accounts  for  the  completely  filled  pixels  and  the  summation  term 
for  the  partially  cloudy  contribution;  the  summation  is  performed  over  all  pixels  classified 
as  partially  cloud-filled.  The  partial  term  is  an  adaptation  of  the  spatial  coherence  energy 
balance  approach  developed  by  Coakley  and  Bretherton  (1982).  The  representative 
radiance  vdues,  Iclr  and  Icld  are  computed  from  the  respective  cloudy  and  clear  brightness 
temperature  cutoff  thresholds. 


5.1.3  Bispectral  Test 

The  OLS  bispectral  algorithm,  developed  for  use  during  daytime  conditions,  is 
similar  to  the  single  channel  algorithm  but  is  applied  in  two  spectral  dimensions 
simultaneously.  Data  from  both  visible  and  infrared  sensor  channels  are  analyzed  using 
two  pairs  of  cutoff  values,  one  pair  for  each  channel.  It  should  be  noted  that  accurate 
specification  of  the  infrared  threshold  is  not  as  critical  in  the  bispectral  test  as  in  the  one 
channel  IR  algorithm  since  low  (warm)  liquid  water  clouds  reflect  well  and  will  generally 
be  detected  from  the  visible  data  when  not  over  highly  reflective  backgrounds. 

Figure  5  provides  an  illustration  of  how  the  two  dimensional  visible-infrared  space 
is  divided  into  nine  classification  regions  by  the  four  cutoff  values.  In  this  figure  Tdd  and 
Tcir  represent  the  infrared  brightness  temperature  cloud  and  clear  cutoff  values  defined  by 
Eqs.  5-3  and  5-4,  respectively,  while  Rdd  and  Rdr  represent  the  visible  count  cloud  and 
clear  cutoffs  defined  by  Eqs.  5-5  and  5-6,  respectively.  Infrared  temperatures  that  are  less 
than  the  infrared  cloud  threshold  value: 

•Tir  <  Tcid 

are  unambiguously  classified  as  cloud-filled.  Data  that  are  both  warm  in  the  infrared  and 
dark  in  the  visible  channel  (Ryis): 

•  Tir  >  Tcid 
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and  •  Ryis  Rcld 

and  'Tir  >  Tcir  or  Ryis  <  Rclr 

are  unambiguously  classified  as  clear.  Warm  bright  regions: 

•Tir  >  Tcid 
and  ‘Ryis  >  Rcld 

require  an  a  priori  dear-scene  classification  to  remove  the  ambiguity  caused  by  the 
similarity  in  radiative  signatures  of  backgrounds  such  as  deserts  and  low  cloud.  Data  that 
fall  between  all  four  threshold  values: 

•  Tcid  <  Tir  <  Tcir 

and  •  Rcld  ^  ^vis  ^  Rclr 

are  classified  as  partially  cloud-filled.  These  rules  are  used  to  define  the  classification  bins 
labeled  in  Figure  5. 

Tcid  Tcir 

Bright 

Rcld 

s 

QQ 

>:  Rclr 

Dark 


Tcid  Tcir 


Cold  Warm 

INFRARED 


Figure  5.  Bispectral  classification  approach. 

Like  the  single  channel  algorithm,  fractional  cloud  amount  is  calculated  based  on  the 
individual  pixel  classifications  within  an  analysis  region.  Again,  contributions  to  cloud 
amount  are  calculated  separately  for  completely  filled  and  partially  filled  FOVs: 


where  Ac  is  effective  cloud  amount,  Ntot  is  the  total  number  of  pixels  in  the  analysis 
region,  Ndd  is  the  number  classified  as  completely  cloud-filled,  Ryis  and  Ijr  are  the 
measured  reflectance  and  calculated  radiance,  respectively,  of  the  partially  cloud-filled 
pixels  and  Idr  and  Idd  are  calculated  from  the  brighmess  temperature  cutoff  values,  Tdr  and 
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Tcid  respectively.  The  Ncidterm  accounts  for  the  contribution  of  cloudy  pixels  and  the 
sununation  is  performed  over  all  pixels  classified  as  partially  filled. 


5.1.4  Use  of  Visible  Data 

Any  algorithm  that  uses  visible  data  requires  filters  for  highly  reflective  terrestrial 
back^ounds.  Three  background  types  or  conditions  have  been  identified  as  problematic 
for  visible  processing:  desert,  snow  and  ice,  and  sun  glint  over  water.  Desert  regions  are 
identified  in  the  geographic  type  supporting  database  (Section  2.7).  No  dynamic 
information  on  snow  cover  is  assumed  to  be  available  on  tactical  terminals.  Possible 
options  for  providing  snow  and  ice  cover  information  to  the  TACNEPH  algorithms  should 
be  considered  since  this  will  directly  impact  the  accuracy  of  the  cloud  analysis.  For 
example,  if  the  tactical  user  suspects  there  is  snow  in  the  analysis  region,  based  on  personal 
observation  or  other  information  sources  (e.g.,  pilot  reports),  a  snow  flag  user  input  could 
be  introduced.  Additionally,  SSM/I  and  AVHRR  data  could  be  used  to  generate  a  separate 
snow  and  ice  cover  supporting  database  that  could  be  input  to  the  TACifePH  model.  A 
sun  glint  test,  based  on  local  satellite  observing  and  solar  geometry,  was  developed  for  use 
over  water  background  surfaces  (as  defined  by  the  geographic  support  database  described 
in  Section  2.7).  Potential  sun  glint  areas  are  defined  by  the  following  test: 

•  Background  surface  type  must  be  water, 

and  •  I  \j/  -  6  I  <  40 , 

and  •  120  <  (j)  <  240 , 

where  the  cutoff  values  are  empirically  set  to  define  the  geographic  extent  over  which  sun 
glint  may  be  expected  to  occur  (see  Section  2.5  for  angle  definitions). 

No  visible  data  processing  is  performed  over  an  analysis  region  that  contains  desert 
or  snow  backgrounds  or  where  the  sun  glint  test  evaluates  as  true.  Also  visible  processing 
is  restricted  to  locations  where  the  sun  is  above  the  horizon  and  not  near  the  terminator. 
These  conditions  are  defined  by  a  solar  zenith  angle  test: 

•  0  >  75. 

The  minimum  zenith  angle  defining  usable  visible  data  is  set  at  this  value  because 
dynamic  onboard  gain  adjustments  to  the  visible  sensor  in  the  vicinity  of  the  terminator 
make  objective  processing  of  the  visible  data  problematic.  In  situations  where  there  is  a 
known  reflective  background  or  the  solar  zenith  angle  is  too  large,  the  OLS  cloud  algorithm 
reverts  to  the  single  channel  IR  test. 

5.1.5  OLS  Cloud  Analysis  Results 

To  support  algorithm  testing  during  the  algorithm  development  process,  the 
research  grade  software  used  to  implement  the  OLS  algorithm  was  configured  to  produce 
pixel  level  output  products  that  contained  results  of  all  the  cloud  tests  available  to  the 
algorithm  (i.e.,  single-channel  visible,  single-channel  IR,  and  bispectral).  Results  are 
encoded  into  an  array  of  8-bit  values  suitable  for  display  as  synthetic  imagery.  Table  6 
provides  definitions  of  the  bit  assignments  for  the  DMSP  Cloud  Analysis  Algorithm 
output.  One  8-bit  quantity  is  produced  for  each  pixel  in  the  input  image,  and  one  file  is 
created  for  each  OLS  input  scene  processed  through  the  OLS  cloud  analysis  algorithm. 
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Table  6.  Intermediate  DMSP  Cloud  Analysis  Bit  Assignments 


Bit 

Assignment 

Description 

0 

Desert 

ON  =  desert  background  type 

1 

Coastline  geography 

ON  =  coastline  background  type 

2 

VIS  partial 

Single-channel  visible  partial  cloud  test 

3 

VIS  total 

Single-channel  visible  total  cloud  test 

4 

IR  partial 

Single-channel  infrared  partial  cloud  test 

5 

ER  total 

Single-channel  infrared  total  cloud  test 

6 

Bispectral  partial 

Bispectral  partial  cloud  test 

7 

Bispectral  total 

Bispectral  total  cloud  test 

In  any  future  operational  implementation  it  is  assumed  that  the  desired  output  will 
be  a  fraction^  cloud  amount  based  on  the  single  most  appUcable  cloud  test.  Applicability 
will  be  determined  by  data  availability.  In  situations  where  both  visible  and  infrared  data 
are  available,  cloud  amount  will  be  determined  by  the  bispectral  test.  Otherwise,  when  data 
from  only  one  sensor  channel  are  available  (e.g.,  nighttime,  failure  of  other  sensor 
channel),  cloud  amount  will  be  determined  by  die  appropriate  single-channel  test. 

5.1.6  OLS  Cloud-Clearing  Procedure 

In  order  to  generate  the  dear-scene  information  required  to  produce  the  supporting 
infrared  corrections  and  visible  background  databases  described  in  Section  4,  it  is 
necessary  to  operate  the  OLS  algorithm  in  a  cloud-cleaiing  mode.  The  cloud-clearing 
procedure  is  similar  to  the  cloud  detection  operation  described  above  except  that  only  the 
clear  cutoff  value  is  used  to  discriminate  clear  from  cloud-contaminated  pixels  and  a  smaller 
threshold  (i.e.,  identifies  warmer  pixels)  is  used  to  define  the  cutoff  value.  The  intent  is  to 
find  and  remove  all  cloud  in  a  scene  with  less  concern  for  overanalysis.  However,  at  the 
same  time  it  is  critically  important  that  at  least  some  clear  areas  are  identified  in  order  to 
update  the  dear-scene  supporting  databases. 

Cloud-clearing  thresholds  are  defined  as  a  percentage  of  the  IR  cloud-detection 
thresholds  and/or  the  visible  clear  cutoff  values.  Real-data  tests  showed  that  different 
thresholds  were  needed  for  different  locations  and  conditions.  Over  the  Canadian  ROI,  the 
IR  threshold  is  30%  smaller  than  the  clear  visible  brightness  count  threshold: 


^Tcid_cir  —  (.70)ATcii- 


(5-9) 


and  over  the  Albany,  Water  and  Georgia  ROI’s  the  reduction  is  15%: 


^Tcld_clr  —  (•85)ATcir  (5-10) 

where  ATdr  is  the  IR  clear  threshold  defined  in  Eq.  5-2.  Infrared  cloud-clearing  cutoff 
values  are  then  defined  by  replacing  ATdr  in  Eq.  5-4  with  ATdd_dr  from  Eq.  5-9  or  5-10: 

Tcld_clr  =  Tpred  -  ATdd_dr  •  (5-1 1) 


Visible  cutoff  values  are  defined  by  reducing  the  clear  cutoff  values  directly,  for  the 
Canadian  ROI  by  30%: 
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Rcld_clr—  (•70)Rclr 


(5-12) 


and  for  the  Albany,  Water  and  Georgia  ROFs  the  percentage  is  15%: 

Rcld_clr=(-85)Rcir  (5-13) 

where  Rcir  is  visible  clear  cutoff  value  defined  in  Eq.  5-6. 

During  the  development  process,  it  became  apparent  that  the  magnitudes  of  the 
cloud-clearing  cutoff  values  are  cracial  to  maintaining  an  accurate  cloud  analysis.  If  no 
clear  areas  are  identified  the  dear-scene  databases  quickly  become  obsolete.  If  some  cloud 
is  misinterpreted  as  clear,  the  background  databases  become  cloud-contaminated  resulting 
in  a  degradation  of  the  cloud  algorithm  accuracy.  To  insure  that  the  cutoff  values  were 
properly  set,  histograms  of  the  dear-scene  differences  between  satellite-observed  and 
climatological  temperatures  were  routinely  generated  and  monitored.  Figure  2  illustrates  a 
typical  dear-scene  temperature  difference  histogram  when  there  is  no  cloud  contamination. 
However,  if  some  cloudy  pixels  remain  in  the  distribution  after  cloud  clearing,  the  resulting 
histogram  assumes  a  bimodal  shape  such  as  that  illustrated  in  Fi^e  6.  When  this  occurs  it 
may  become  necessary,  if  the  accuracy  of  the  cloud  analysis  begins  to  degrade,  to  manually 
adjust  the  2a  limits  (ATminj  and  ATmaxi)  defined  by  Eqs.  4-7  and  4-8  to  remove  the 
contaminated  pixels  from  the  dear-scene  statistics.  Failure  to  correct  for  cloud- 
contaminated  pixels  can  result  in  a  negative  feed-back  process  wherein  the  time  and 
frequency-weighted  minimum  AT  limit  from  Eq  4-10  tecomes  too  small,  allowing  some 
cloud-contaminated  pixels  to  satisfy  the  inequality  in  Eq.  4-12  resulting  in  a  predicted  dear- 
scene  temperature  (Tpredn  from  Eq.  4-13)  that  may  be  umealistically  low.  If  this  Tpred  value 
is  used  in-tum  in  Eq.  5-1 1  to  detect  and  eliminate  cloud-contaminated  pixels,  more  cloudy 
pixels  can  be  incorrectly  classified  as  cloud-free  and  the  process  builds  on  itself. 


AT, 


nun 


AT 


max 


Tsat  ■  Tdim 


Figure  6.  Example  histogram  of  comparison  between  satellite  brightness  tempera¬ 

tures  and  corresponding  climatological  temperatures  for  a  case  in  which 
some  cloudy  pixels  have  been  incorrectly  classified  as  cloud-free.  Solid 
vertical  lines  represent  2  standard  deviations  about  the  mean  of  the 
entire,  bimodal  distribution.  Broken  vertical  lines  represent  manually 
adjusted  values  for  which  the  cloud  contaminated  pixels  have  been 
eliminated. 
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5.2  NOAA/AVHRR  Cloud  Analysis  Algorithm  Description 


The  TACNEPH  AVHRR  cloud  detection  algorithm  consists  of  a  series  of  tests, 
comprised  of  cloud  and  background  surface  classification  tests,  each  one  of  which  relies  on 
one  or  a  combination  of  separate  cloud  spectral  signatures.  The  approach  is  based  on  work 
done  by  Saunders  and  Kriebel  (1988).  The  background  tests  are  used  to  identify 
problematic  background  surface  conditions  that  can  cause  the  cloud  tests  to  misclassify  the 
dear-scene  as  cloudy  (e.g.  snow/ice,  sun  glint  and  desert)  (Section  5.2.1).  Results  of 
these  tests  are  used  to  modify  affected  cloud  tests  and/or  eliminate  channels  from  the 
analysis  process.  The  algorithm  is  structured  to  run  each  of  the  tests  and  store  the 
individud  results  in  a  temporary  internal  buffer  termed  the  intermediate  cloud  analysis 
(Section  5.2.4).  It  is  important  to  note  that  some  of  the  tests  require  the  results  of  other 
tests  so  the  order  in  which  the  tests  are  ran  is  important.  Noise  effects  resulting  from 
problems  in  the  AVHRR  channel  3  sensor  are  removed  from  specific  cloud  test  results 
using  a  noise  filter  (Section  5.2.3).  After  all  tests  have  been  completed,  data  stored  in  this 
internal  buffer  are  analyzed  concurrently  to  produce  the  final  cloud  analysis  product 
(Section  5.2.5).  For  the  purposes  of  the  algorithm  development  effort,  cloud  analysis 
results  are  output  on  a  pixel  by  pixel  level.  Depending  on  the  specific  implementation  and 
database  requirements  on  the  tactical  terminal,  these  results  can  be  easily  reformatted  to 
either  a  Polar  Stereographic  or  Mercator  map  projection  at  the  desired  grid  resolution.  It 
may  be  useful  to  note  that  for  the  research  grade  implementation,  intermediate  results  for 
each  of  the  separate  tests  were  output  and  visually  examined.  This  information  was 
extremely  useful  as  a  diagnostic  for  algorithm  performance  and  to  provide  additional  insight 
into  specific  types  of  cloud  (e.g.,  certain  tests  are  particularly  sensitive  to  low  cloud  or 
cirrus)  (Section  5.3.1).  If  this  type  of  information  should  be  determined  to  be  useful  in  the 
operational  implementation,  it  could  be  easily  produced  through  proper  formatting  of  the 
internal  buffer  referred  to  above. 

5.2.1  Background  Surface  Filter  Tests 

Background  surface  tests  are  used  to  identify  problematic  surface  backgrounds  that 
have  spectral  signatures  similar  to  cloud.  Results  of  Aese  tests  are  used  to  either  modify 
affected  cloud  tests  or  eliminate  channels  from  the  final  analysis  process.  Table  7 
summarizes  the  background  tests,  the  sensor  channels,  and  die  solar  zenith  angle 
requirements. 


Table  7.  Background  Surface  Test  Summary 


Name 

Description 

AVHRR 

Channel 

Solar  Aenith 
Angle 

SNOW 

snow  and  ice  detection 

1,2,3,4 

0  <  80' 

GLINT 

detection  of  sun  glint  over  large  bodies  of 

water 

1,2,3,4,5 

e<80° 

DESERT 

highly-reflective  non-vegetated  land 

surfaces 

2,3,4 

0<8O° 
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5.2. 1.1  Sun  Glint  Test 


The  sun  glint  test  (GLINT)  is  used  to  detect  specular  reflection  off  of  water  surfaces 
which  could  be  mistakenly  identified  as  cloud  by  tests  that  rely  on  reflected  solar  radiation. 
Sun  glint  is  a  potential  problem  over  any  water  surfaces  that  can  be  resolved  by  the  satellite, 
however,  in  practice  the  GLINT  test  is  only  applied  over  permanent  water  surfaces  large 
enough  to  be  captured  in  the  geography  database  (Section  2.7).  In  contrast  to  the  OLS 
algorithm  described  above,  the  AVHRR  sun  glint  test  uses  spectral  information  along  with 
geometry  considerations  to  classify  glint  conditions.  There  is  no  sun  glint  signature  that  is 
distinct  from  all  cloud  signatures;  therefore  the  GLINT  algorithm  relies  on  a  series  of 
signatures,  any  one  of  which  could  be  representative  of  either  cloud  or  glint,  but  when 
taken  together  greatly  enhance  the  probability  that  the  cause  is  glint  rather  than  cloud.  This 
has  the  additional  benefit  of  discriminating  cloud  within  a  sun  glint  region,  eliminating  the 
need  to  remove  glint  areas  from  visible  chaimel  cloud  processing.  A  series  of  conditions 
involving  the  background,  solar/satellite  geometry,  and  spectral  signatures  must  be  met  to 
detect  glint.  All  tests  must  evaluate  positively  for  glint;  the  failure  of  any  single  test 
prevents  a  glint  classification. 

The  first  set  of  tests  determine  if  the  background  surface  type  and  solar/satellite 
geometry  will  support  sun  glint.  These  tests  are: 

•  Background  surface  type  must  be  water , 

and  •l\|r-ei  <  40°, 

and  •  120°  <  (j)  <  240° , 

where  \|r  is  the  sateUite  zenith  angle  and  0  is  the  solar  zenith  angle,  and  ^  is  the  azimuth 
angle  between  the  sun  and  the  satellite.  The  threshold  values  are  empirically  derived. 

The  second  set  of  tests  examines  the  spectral  signature  of  any  pixels  that  passed  the 
background  surface  type  and  solar/satellite  geometry  tests.  These  tests  are: 

The  albedo  must  be  high  in  the  visible  channels: 

•  BRIGHT  Test  detects  cloud  (Section  5.2.2.2  ) , 

or  •  RATIO  Test  detects  cloud  (Section  5.2.2.2). 

Channel  3  must  be  nearly  saturated: 

•T3  >  309K, 

and  •  T3  >  T4  +  14  K. 

The  IR  brightness  temperature  must  be  relatively  high  in  the  infrared  chaimels  (i.e., 
not  indicative  of  a  low  liquid  water  cloud): 

•  COLD  Test  does  not  detect  cloud  (Section  5.2.2. 1) , 

and  •  CIRRUS  Test  does  not  detect  cloud  (Section  5.2.2. 1). 

If  all  of  these  conditions  are  satisfied,  the  pixel  is  classified  as  sun  glint  and  bit  0  of 
the  intermediate  output  word  is  set. 


38 


5.2.1.2  Desert  Test 


The  Desert  Test  (DESERT)  is  used  to  augment  the  geographic  database  (Section 
2.7)  through  identification  of  dear-scene  desert  backgrounds  by  analysis  of  multispectral 
daytime  AVHRR  data.  In  this  application,  the  term  desert  is  used  to  indicate  any  Wghly 
reflective,  non-vegetated  land  surface;  it  does  not  necessarily  follow  the  geographer's 
definition  based  on  annual  precipitation.  The  desert  geography  was  developed  % 
thresholding  dear-scene  visible  imagery.  This  works  well  for  identifying  stable  surface 
features  but  it  is  restricted  to  the  reflectance  information  in  the  scenes  used  to  develop  the 
background  classification.  It  cannot  account  for  daily  and  long  term  changes  in  reflectance 
resulting  from  the  angle  of  solar  illumination  or  seasonal  effects,  for  example.  A  dynamic 
test,  which  classifies  the  background  based  on  the  information  in  the  current  scene  proved 
necessary.  Also,  in  addition  to  their  run-time  use  in  specifying  cloud-free  desert  pixels  in 
the  AVHRR  algorithm,  desert  flags  are  potentially  useful  to  the  DMSP/OLS  cloud  analysis 
algorithms  as  a  high  resolution  source  for  identifying  bright  sandy  backgrounds.  The 
DESERT  test  combines  a  series  of  spectral  signatures  as  follows: 

In  addition  to  the  pixel  being  a  land  point,  a  series  of  five  AVHRR  spectral 
conditions  must  be  met  in  order  to  classify  a  pixel  as  cloud-free  desert.  The  first  condition 
is  a  modified  version  of  the  RATIO  cloud  test  (Section  5.2.2.2): 

•0.85  <  A2/A1  <  1.05, 

where  the  empirically  determined  thresholds  bound  the  range  of  the  near-IR  to  visible 
channel  ratio  for  dear-scene  desert  backgrounds.  Note  these  thresholds  are  more  limiting 
than  the  cloud  detection  RATIO  thresholds  since  highly  reflective  land  surfaces  generally 
do  not  exhibit  as  much  variability  as  clouds  in  near-IR  and  visible  sensor  channel 
measurements. 

The  second  test  is  an  absolute  check  on  the  chaimel  2  albedo  testing  for  a 
(potentially)  clear  background.  The  test  is  defined  as: 

•A2  <  0.2, 

and  is  employed  to  ensure  the  measured  albedo  is  not  large  enough  to  be  a  cloud  signature 
since  desert  surfaces  are  not  as  bright  as  cloud  in  channel  2. 

The  third  test  determines  if  the  channel  3  brightness  temperature  is  near  samration. 
Clear  non-vegetated  surfaces  exhibit  a  strong  solar  component  in  channel  3  resulting  in  a 
large  T3  brighmess  temperature.  The  test  is  defined  as: 

•T3  >  300  K, 

The  fourth  test  checks  for  a  T4  temperature  that  is  warmer  than  all  but  low  cloud: 
•T4  >290K, 

The  final  desert  criterion  ensure  that  low  clouds  do  not  get  classified  falsely  as 
desert  by  restricting  the  brighmess  temperature  difference  between  channels  3  and  4  to  a 
specified  range.  The  test  is  defined  as: 

•  7.0  K  <  T3  -  T4  <  17.0  K. 
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If  all  of  these  conditions  are  satisfied,  the  pixel  is  classified  and  bit  0  of  the  intermediate 
ou^ut  word  is  set.  Note  that  the  bit  assignment  of  this  test  does  not  conflict  with  the 
GLINT  test  as  DESERT  is  applied  only  over  land  surfaces. 


5.2. 1.3  Snow  Background  Test 

The  SNOW  test  is  used  to  discriminate  snow  and  ice  backgrounds  from  cloud.  It  is 
a  two  tiered  algorithm;  first  it  uses  multispectral  data  to  identify  scenes  with  characteristics 
consistent  with  snow  (but  not  necessarily  separate  from  cloud).  The  second  step  provides 
the  principal  discrimination  signature  for  separating  snow  from  cloud  based  on  die  MWIR 
reflectivity.  At  the  3.7  /tm  wavelength,  water  droplet  clouds  tend  to  be  reflective  while 
snow  backgrounds  are  not.  Thus,  if  the  scene  is  characteristic  of  either  snow  or  cloud,  but 
is  not  reflective  in  channel  3,  then  it  is  classified  as  snow.  To  establish  the  basic  scene 
characteristics  a  series  of  tests  are  applied: 

The  first  two  tests  ensure  that  the  temperature  is  close  to  fireezing  and  close  enough 
to  the  surface  temperature  not  to  be  cloud, 

•T4<278  K; 

and  •  I  T4  -  Tpred  I  <  15 


where  Tpred  is  defined  in  Eqs.  4-13  and  4-14.. 

The  next  tests  look  for  reflection  in  the  visible(over  land)  or  near-IR  (over  water)  channels 

•  A2  >  0.24  over  water  or  Aj  >0.1  over  land. 

Then,  if  the  data  fall  within  all  of  these  limits,  the  final  test  is  applied: 

•  I  T3  -  T4  I  <  5. 

If  snow  is  detected,  then  bit  7  of  the  intermediate  output  word  is  set.  Note  that  if 
this  spectral  snow  test  evaluates  as  true,  the  pixel  is  unambiguously  classified  as  cloud-firee 
and  results  from  other  tests  are  not  used.  Testing  on  real  data  has  shown  that  cloud  in 
cloud  shadows  exhibit  spectral  signatures  that  are  very  similar  to  snow  and  may  be 
misclassified  as  clear.  Stricter  thresholds  can  reduce  this  misclassification;  however, 
correct  snow  classifications  are  also  reduced.  The  cloud  shadow  effect  is  minimal 
compared  with  the  problems  resulting  from  snow  so  this  error  is  accepted. 

5.2.2  Cloud  Tests 

Recall  from  Section  5  that  each  AVHRR  cloud  detection  test  is  based  on  a  spectral 
signature  that  exploits  one  or  more  of  the  AVHRR  sensor  channels.  Each  cloud  test  is 
designed  to  be  more  sensitive  to  a  specific  type  of  cloud;  there  is  no  one  test  that  can 
identify  all  types  of  cloud  under  all  conditions.  In  total,  there  are  seven  tests  which  are 
divided  into  toee  groups:  1)  those  that  are  equally  applicable  without  regard  to  the  amount 
of  solar  illumination  on  the  scene;  2)  daytime  tests  that  rely  on  reflected  solar  radiation;  and 
3)  nighttime  tests  that  are  applicable  only  in  the  absence  of  sunlight.  A  solar  zenith  angle 
threshold  is  used  to  determine  which  tests  are  applicable  to  a  particular  situation.  Daytime 
is  defined  as  a  solar  zenith  angle  less  than  or  equal  to  90  degrees;  Nighttime  is  defined  as  a 
solar  zenith  angles  greater  than  90  degrees.  Some  of  the  daytime  tests  require  even  stricter 
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solar  zenith  angle  thresholds.  These  restrictions  will  be  noted  in  the  appropriate  test 
descriptions. 

Each  pixel  in  a  scene  is  subjected  to  all  applicable  cloud  tests.  Data  from  all  five 
AVHRR  channels,  the  background  surface  tests  (Section  5.2.1),  and  the  supporting  dear- 
scene  information  (Section  4)  are  required  to  analyze  the  scene.  Table  8  summarizes  the 
cloud  tests  and  the  solar  zenith  angle  requirements  that  dictate  when  they  are  applied. 

Table  8.  AVHRR  Cloud  Test  Summary 


Name 

Description 

Channel 

Solar  Zenith 
Angle 

COLD 

single  LWIR  channel  threshold 

4 

NA 

CIRRUS 

split  LWIR 

4,5 

NA 

BRIGHT 

single  vis  or  near-IR  channel  threshold 

1,2 

e<6oo 

RATIO 

comparison  of  near-IR  to  visible  albedo 

1,2 

0<8OO 

CLF 

difference  between  MWIR  and  LWIR 

3,4 

0<9OO 

FLS 

difference  between  LWIR  and  MWIR 

3,4 

0>9OO 

HIGH 

difference  between  MWIR  and  LWIR 

3,4,5 

0>9OO 

5.2.2. 1  Solar  Independent  Cloud  Tests 

There  are  two  tests  used  to  detect  cloud  that  are  independent  of  the  scene  solar 
illumination.  The  first  is  a  single  channel  LWIR  threshold  (COLD)  test  designed  to  detect 
mid-level  water  droplet  clouds,  optically  thick  cirrus,  and  any  other  cloud  that  is  thermally 
distinct  from  the  terrestrial  background.  The  second  (CIRRUS)  uses  the  split  LWIR 
channels  to  detect  thin  cirms  and  small  ice  or  water  particles  at  the  edge  of  mid-level  cloud. 

The  COLD  test  is  similar  to  the  RTNEPH  satellite  processor  and  OLS  single 
channel  algorithm  in  that  it  is  a  single  channel  IR  threshold  test  that  attempts  to  separate  out 
the  thermal  signature  of  clouds  from  an  estimated  terrestrial  background  signature.  The 
TACNEPH  algorithm  uses  the  surface  temperature  climatology  database  (Section  2.6) 
along  with  the  internally  generated  infrared  dear-scene  statistics  database  (Section  4.2)  to 
predict  the  dear-scene  channel  4  radiative  temperature.  For  the  research  implementation, 
data  were  processed  over  a  32  x  32  pixel  analysis  array  and  segregated  by  background  type 
into  two  classes:  1)  land  backgrounds  and  2)  water  backgrounds  (in  the  tactical  terminal 
implementation  the  32  x  32  array  would  be  replaced  by  the  desired  analysis  region).  For 
each  background  type,  channel  4  data  are  organized  in  a  histogram  from  coldest  to  warmest 
and  a  reference  temperature  value  is  identified  for  which  5%  of  the  data  in  the  distribution 
are  warmer.  A  warm  temperature  is  selected  to  maximize  the  probability  that  a  cloud-free 
location  will  be  selected  for  the  reference  temperature.  The  5%  limit  is  added  to  eliminate 
anomalously  warm  pixels  from  consideration.  A  pixel  with  a  brightness  temperature  equal 
to  the  reference  temperature  is  located  in  the  32  x  32  analysis  array  and  the  corresponding 
climatological  surface  temperature  is  calculated  from  the  climatology  database.  As  is  the 
case  when  generating  the  dear-scene  statistics  database,  the  climatological  temperature  is 
obtained  by  first  collocating  the  sensor  and  climatology  data  and  then  time  interpolating  the 
climatological  temperatures  to  match  the  satellite  valid  times.  The  time  interpolation  is  not 
critical  to  the  dear-scene  temperature  estimation  process  except  that  it  tends  to  eliminate 
artificial  boundaries  in  the  analysis  when  the  sensor  data  passes  from  one  climatological 
time  regime  to  another  (e.g.,  from  one  synoptic  time  to  another  or  over  the  end  of  one 


41 


month  into  the  next).  Once  the  reference  pixel  is  identified  and  a  climatological  surface 
temperature  is  calculated,  a  predicted  dear-scene  brightness  temperature  is  calculated  for  the 
analysis  region  using  the  process  described  in  Section  4.2. 

A  cloud  decision  is  performed  by  comparing  each  T4  pixel  value  to  its  correspond¬ 
ing  predicted  dear-scene  temperature: 


Tpred  ■T4  >  THRESHcold. 


If  the  T4  value  is  less  than  the  Tpred  value  by  an  amount  greater  than  a  preset  threshold, 
then  the  pixel  is  declared  cloudy  and  bit  3  in  the  intermediate  output  word  is  set.  The 
magnitude  of  the  threshold  was  established  empirically  as  a  measure  of  the  uncertainty  in 
the  dear-scene  temperature  prediction.  Separate  threshold  values  are  maintained  as  a 
function  of  the  background  type  including  land,  water,  desert,  snow,  and  coast.  Current 
THRESHcold  values  are:  9  K  for  water  backgrounds,  10  K  for  land,  10  K  for  desert,  15 
K  for  snow,  and  25  K  for  coastline.  Cloud  clearing  thresholds  are  0  K  for  water 
backgrounds,  0  K  for  land,  10  K  for  desert,  5  K  for  snow,  and  25  K  for  coastline. 

Accurate  detection  of  clouds  over  coastlines  has  proven  particularly  difficult  for  the 
COLD  test.  Over  coastlines,  there  can  be  a  large  temperature  gradient  between  the  land  and 
water  surfaces.  Since  the  reference  temperature  is  representative  of  the  warmer  surface,  the 
colder  surface  may  be  misclassified  as  cloud.  This  problem  can  be  avoided  by  stratifying 
the  background  points  into  land  and  water,  selecting  two  reference  temperatures,  and 
applying  separate  corrections  to  land  and  water  points.  However,  the  coarse  resolution  of 
the  geography  database  has  made  it  difficult  to  correctly  stratify  the  background  points.  In 
addition  to  coastiines,  this  problem  occurs  over  land  areas  containing  lakes  which  are  too 
small  to  be  reflected  in  the  geography  database.  If,  for  example,  the  lake  is  warmer  than 
the  surrounding  land,  the  reference  temperature  will  be  representative  of  the  lake  and  the 
surrounding  land  will  be  misclassified  as  cloudy.  A  higher  resolution  geography  database 
with  specific  emphasis  on  accurate  placement  of  coastline  would  eliminate  many  of  these 
surface  classification  problems  allowing  the  5%  limit  imposed  to  eliminate  anomalously 
warm  pixels  from  selection  as  the  reference  temperature  to  be  reduced.  A  higher 
resolution,  64*  mesh  geography  database  was  developed  and  used  successfully  to  address 
this  problem  as  part  of  the  parallel  SERCAA  research  program  (Gustafson  et  aJ.,  1994). 

The  CIRRUS  test  exploits  the  sensitivity  of  the  channel  5  radiance  to  the  presence 
of  small  ice  crystals  present  in  thin  cirrus  and  small  water  droplets  generally  found  around 
the  edge  of  water  clouds.  The  cloud  signature  derives  from  the  channel  4-5  difference 
since  the  emissivity  at  channel  4  wavelengths  is  relatively  insensitive  to  the  presence  of 
these  particles.  However,  care  must  be  taken  when  implementing  this  test  since  chaimel  5 
radiance  is  also  affected  by  the  presence  of  water  vapor  in  the  atmosphere.  Thus  the  split 
window  difference  test  uses  a  theoretically  derived  table  of  expected  differences  between 
T4  and  T5  that  is  maintained  as  a  function  of  expected  water  vapor  loading  and  path  length 
through  the  atmosphere.  Saunders  and  Kriebel  (1988)  derived  such  a  table  for  the  NorA 
Atlantic  Ocean  region  that  was  found  to  be  generally  applicable  during  TACNEPH  testing. 
Since  no  direct  humidity  measurement  is  assumed  to  be  available  in  a  tactical  terminal,  the 
predicted  dear-scene  temperature  (Tpred)  is  used  as  a  surrogate  for  water  vapor  (i.e.,  the 
higher  the  surface  temperature  the  greater  amount  of  water  vapor  that  can  be  maintained). 
Table  9  contains  the  expected  T4-T5  values  used  in  the  CIRRUS  test.  A  cloud  detection 
threshold  is  established  by  interpolating  to  the  actual  T4  and  satellite  zenith  angle  (\|/) 
values.  The  CIRRUS  test  is  defined  as: 

•  T4  -  T5  >  THRESH(T4,\|/) 
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where  THRESH(T4  \|/)  is  the  cloud  detection  threshold  obtained  through  interpolation  from 
the  table.  If  the  T4-t5  difference  exceeds  the  threshold  level,  the  point  is  declared  cloudy 
and  bit  2  in  the  intermediate  output  word  is  set.  There  are  no  separate  cloud  clearing 
thresholds  used  in  this  test. 

Table  9.  Predicted  Clear-Scene  Channel  4-5  Brightness  Temperature  Differences 

(from  Saunders  and  Kriebel,  1988) 


TpredW 

0 

36 

48 

55 

60 

260 

0.55 

0.60 

MM 

0.90 

1.10 

270 

0.58 

0.63 

1.03 

1.13 

280 

1.30 

1.61 

1.88 

2.14 

2.30 

290 

3.06 

3.72 

3.95 

4.27 

4.73 

300 

5.77 

6.92 

7.00 

7.42 

8.43 

310 

9.41 

10.74 

11.03 

11.60 

13.39 

Real-data  tests  have  shown  that  the  CIRRUS  test  performs  accurately  and  robustly 
for  the  majority  of  climatological  situations.  However,  the  test  sometimes  has  difficulty 
accurately  discriminating  cirrus  cloud  from  snow  and  ice  backgrounds.  An  improved 
CIRRUS  test,  which  attempts  to  correct  this  problem,  ideally  requires  snow  and  ice  field 
information.  However,  snow  information  is  currently  unavailable  to  the  TACNEPH 
program  other  than  through  the  spectral  SNOW  test.  Recall  that  if  the  spectral  SNOW  test 
identifies  snow,  the  pixel  is  unambiguously  classified  as  clear.  In  TACNEPH,  a  surface 
temperature  dependency  is  used  to  identify  potentially  snow  covered  surfaces: 

*  Tpred  <  280  K. 


If  this  criterion  is  met,  an  additional  requirement  is  placed  on  the  cloud  test.  Based  on  the 
assumption  that  channel  4  brightness  temperatures  measured  from  cirrus  clouds  are  colder 
than  the  terrestrial  background,  the  T4  brightness  temperature  is  required  to  be  lower  than 
the  predicted  dear-scene  brightness  temperature  by  an  amount  greater  than  a  cloud  detection 
threshold.  This  test  is  defined  as: 

•  Tpred  ■  T4  >  5  K. 


If  the  background  is  potentially  snow  or  ice  and  both  cloud  criteria  are  met,  then  the  pixel 
is  classified  as  cloud-filled  and  bit  2  in  the  intermediate  output  word  is  set.  If  the 
background  is  not  classified  as  snow  or  ice,  then  only  the  T4  -  T5  test  is  required. 

Should  a  snow  database  become  available  to  TACNEPH  in  the  future,  snow  and  ice 
covered  surfaces  should  be  identified  using  the  database  rather  than  the  surface  temperature 
dependency. 


S.2.2.2  Daytime  Cloud  Tests 

Three  cloud  tests  are  used  during  daytime  passes  that  rely,  at  least  partially,  on 
reflected  solar  radiation  for  a  cloud  signature.  The  first  is  used  to  detect  water  droplet 
clouds  that  exhibit  a  high  albedo  relative  to  the  terrestrial  background  (BRIGHT);  the 
second  uses  the  ratio  of  near-IR  to  the  visible  albedo  to  identify  signatures  that  depart  from 
expected  cloud-free  land  and  water  background  signatures  (RATIO);  and  the  third  contrasts 
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the  brightness  temperature  derived  from  both  the  emitted  and  reflected  components  of  the 
MWIR  channel  for  large  water  droplet  clouds  to  the  emitted  only  component  at  longer 
wavelengths  (CLF).  However,  these  signatures  are  not  unique  to  cloud  and  the  tests  can 
be  confused  by  terrestrial  backgrounds  Aat  exhibit  similar  reflectance  characteristics 
including  snow,  desert,  and  sun  glint.  To  discriminate  cloud  from  these  backgrounds,  the 
background  surface  filter  tests  described  in  Section  5.2.1  are  used  to  modify  the  actual 
cloud  test  thresholds  or  filter  ambiguous  information  from  the  cloud  test  results. 

The  BRIGHT  test  is  a  single  channel  threshold  algorithm  that  attempts  to 
discriminate  relatively  bright  cloud  reflectance  from  a  known  background.  The  test 
requires  both  channel  1  and  2  data  along  with  the  internal  dear-scene  albedo  database.  The 
sensor  data  are  normalized  to  remove  the  effects  of  anisotropic  reflection  in  exactly  the 
same  way  that  the  dear-scene  albedo  database  was  generated  (Section  4.1).  These  data  are 
only  processed  out  to  a  solar  zenith  angle  of  60  degrees  since  ARF  data  are  not  available 
beyond  that  range  and,  in  any  event,  it  is  questionable  whether  there  is  sufficient  solar 
illumination  to  obtain  a  good  cloud  signature.  The  normalized  sensor  albedo  data  are 
compared  to  the  stored  dear-scene  albedo  data  for  the  collocated  grid  box  and,  if  the  sensor 
data  exceeds  the  stored  value  by  an  amount  exceeding  a  preset  threshold,  the  pixel  is 
classified  as  cloudy  and  bit  6  of  the  intermediate  output  word  is  set.  The  threshold  was 
empirically  defined  as  0.16  over  water  surfaces  and  0.07  over  land.  Channel  2  data  are 
used  over  water  backgrounds  to  exploit  the  relative  transparency  of  the  atmosphere  relative 
to  the  visible  data.  However,  over  land  surfaces  channel  1  data  are  used  because  vegetated 
surfaces  exhibit  a  relatively  higher  reflectivity  in  the  near-IR  that  normally  outweighs 
atmospheric  transmission  effects.  In  both  cases  the  goal  is  to  use  the  sensor  data  that 
minimize  contributions  from  sources  other  than  cloud. 

The  RATIO  test  is  unique  in  that  it  attempts  to  detect  a  visible  and  near-IR  signature 
that  is  not  representative  of  a  clear  terrestrial  background  rather  than  explicitly  searchung  for 
a  cloud  signature.  As  discussed  above,  visible  channel  data  are  relatively  more  susceptible 
to  brightening  from  atmospheric  scattering,  particularly  from  suspended  aerosols  compared 
to  the  near-IR  channel  data.  However,  vegetated  land  surfaces  exhibit  a  reverse  trend  due 
to  a  higher  reflectivity  in  the  near-IR  compared  to  visible  channel  wavelengths.  The 
RATIO  test  exploits  these  characteristics  by  comparing  the  relative  brightness  of  the 
channel  1  and  2  data  through  a  channel  ratio.  No  normalization  for  anisotropic  effects  is 
needed  since  they  cancel  in  the  ratio  operation.  The  test  is  applied  by  simply  dividing  the 
measured  channel  2  albedo  by  the  channel  1  value.  Vegetated  land  surfaces  tend  to  have  a 
ratio  considerably  greater  than  1;  water  surfaces  will  be  less  than  1  depending  on  the 
transmissivity  of  the  atmosphere.  Clouds  exhibit  a  ratio  approximately  equal  to  1.  Thus 
the  cloud  test  is  applied  by  testing  whether  the  channel  ratio  is  approximately  1.  Two  sets 
of  cutoff  values  are  used  depending  on  the  perceived  near-surface  humidity  level.  The 
predicted  dear-scene  temperature  (Tpred)  is  used  to  identify  potentially  high  humidity 
regions  that  could  enhance  aerosol  growth. 

If  the  predicted  temperature  is  less  than  or  equal  to  295  K  the  cloud  detection  test  is  defined 
as: 


•  0.75  <  A2/Ai<  1.1  (cloud  clearing  0.7, 1.2 ); 

if  warmer  than  295  K  then  the  limits  are  changed: 

•  0.70  <  A2/Ai<  1.0. 

If  the  channel  ratio  falls  within  the  appropriate  limits  the  pixel  is  classified  cloudy  and  bit  5 
of  the  intermediate  output  word  is  set.  Real-data  tests  have  shown  that  this  test  can  falsely 
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detect  cloud  over  coastlines.  It  is  believed  that  the  signatures  from  the  mixed  land-water 
field  of  view  result  in  a  spectral  signature  ratio  which  is  close  to  1.  To  eliminate  false 
detection  of  cloud,  this  test  is  not  used  for  any  data  for  which  the  background  is  identified 
as  coast  by  the  geography  database  (Section  2.7).  Coastline  which  is  not  identified  in  this 
coarse  resolution  geography,  e.g.  small  lakes  and  rivers,  remains  a  problem. 

Because  the  BRIGHT  and  the  RATIO  tests  rely  on  the  reflectance  characteristics  of 
cloud  to  identify  cloud,  reflective  terrestrial  backgrounds  (i.e.  snow  or  ice,  desert  and  sun 
glint  backgrounds)  that  exhibit  a  visible  or  near-IR  signature  similar  to  cloud  can  be 
incorrectly  identified  as  cloud.  To  minimize  over-andysis  of  cloud,  these  test  results  are 
not  used  when  the  background  has  been  identified  as  snow,  ice,  desert  or  sun  glint  by  the 
background  surface  tests  (Section  5.2.1)  or  the  geography  database  (Section  2.7). 

The  CLF  test  also  relies  on  reflected  solar  to  detect  low  water  clouds  and  fog.  The 
cloud  signature  is  based  on  the  different  radiative  characteristics  of  small  water  droplet 
clouds  at  channel  3  and  4  wavelengths.  During  daylight  conditions,  the  cloud  signature  in 
channel  3  is  a  combination  of  both  emitted  and  reflected  solar  energy.  At  the  longer 
chapel  4  wavelength  there  is  only  an  emitted  component.  For  most  other  surfaces,  the 
radiative  signatures  for  both  channels  are  approximately  equal  (e.g.  vegetated  land  is  not 
reflective  in  channel  3).  The  result  of  these  characteristics  is  that  brightness  temperatures 
calculated  from  channel  3  radiance  measurements  that  contain  cloud  are  much  larger  than 
for  chaimel  4.  The  CXF  test  is  applied  as  a  channel  brightness  temperature  difference:  if 
T3  .  T4  is  greater  than  a  threshold  then  cloud  is  detected.  The  magnitude  of  the  threshold  is 
defined  empirically  as: 


•  T3  -  T4  >  12  K  (cloud  clearing  4  K) 

under  normal  conditions.  However,  since  this  test  relies  on  the  reflected  component  in 
channel  3,  any  surface  features  other  than  cloud  that  reflect  well  at  3.7  jum  may  result  in 
incoffect  cloud  classification.  Sun  glint  and  non- vegetated  land  surfaces  such  as  desert  fall 
in  this  category;  the  CLF  test  must  be  modified  to  handle  these  surface  conditions.  Snow  is 
generally  not  a  problem  as  it  does  not  reflect  well  at  3.7  jjm.  The  GLINT  test  described 
above  (Section  5.2.1)  is  not  sufficient  to  filter  these  test  results  because  the  CLF  test  is 
more  sensitive  to  low  levels  of  sun  glint  than  either  the  BRIGHT  or  RATIO  tests.  Since 
the  level  of  sun  glint  that  will  affect  this  test  cannot  be  detected  directly,  additional  surface 
type  and  geometric  conditions  are  applied  to  eliminate  all  conditions  that  can  potentially 
support  glint.  These  conditions  are  identical  to  the  geometric  requirements  for  the  GLINT 
test: 

•  Background  surface  type  must  be  water , 
and  •  I  \|r  -  0  I  <  40° , 

and  •  120°  <  ^  <  240° . 

If  these  conditions  are  met,  then  the  CLF  threshold  is  increased: 

•T3-T4>54K  (cloud  clearing  4  K) 

requiring  clouds  to  be  extremely  bright  in  channel  3.  The  CLF  threshold  is  also  raised  for 
non-vegetated  land  backgrounds  which  are  identified  either  through  the  spectral  DESERT 
test  (Section  5.2.1)  or  the  geography  database  (Section  2.7): 

if  ‘DESERT 
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then 


T3  -  T4  >  20  K  (cloud  clearing  20  K). 


The  desert  threshold  does  not  need  to  be  as  high  as  the  sun  glint  threshold  because  desert 
reflectance  is  not  as  high  as  sun  glint  in  channel  3.  If  the  T3  -  T4  difference  exceeds  the 
appropriate  threshold,  the  pixel  is  classified  cloudy  and  bit  4  of  the  intermediate  output 
word  is  set. 


5.2.2.3  Night  Tests 

The  two  remaining  cloud  tests  can  only  be  used  in  the  absence  of  solar  illumination. 
Both  tests  use  channel  3  data  and  would  be  adversely  affected  by  the  presence  of  reflected 
solar  radiation  in  the  channel  3  measurements.  The  first,  FLS,  is  the  nighttime  equivalent 
of  the  CLF  test,  and  again  exploits  the  difference  in  radiative  characteristics  of  small  water 
droplet  clouds  between  the  mid  wave  and  long  wave  IR  bands.  The  second  test,  HIGH, 
exploits  the  non  linearity  of  the  Planck  function  between  the  channel  3  and  channel  4 
bandwidth  to  detect  optically  thin  cirrus. 

The  FLS  test  requires  channel  3  and  4  brightness  temperature  data  and  is  used  to 
detect  low  cloud  and  fog  at  night.  Low,  nighttime  clouds  tend  to  have  a  lower  emissivity  at 
channel  3  wavelengths  relative  to  the  longer  wavelength  channel  4.  This  results  in  a 
slightly  lower  channel  3  brightness  temperature,  often  less  than  1  K  difference.  The  FLS 
test  compares  the  T4  -  T3  difference  against  a  preset  threshold: 

•  T4  -  T3  >  1 .0  (cloud  clearing  0.6  K), 

if  the  difference  exceeds  the  threshold,  bit  4  of  the  intermediate  ouq)ut  word  is  set.  Real- 
data  tests  over  desert  surfaces  commonly  exhibited  an  over-analysis  of  cloud;  this  was 
corrected  by  raising  the  threshold  for  all  data  identified  as  desert.  For  this  test,  desert  is 
identified  in  the  geography  database  (Section  2.1.3)  as  the  DESERT  test  is  only  applicable 
under  daytime  conditions: 

if  •  desert 

then  •  T4  -  T3  >  2.0  (cloud  clearing  0.6  K) . 

Because  the  cloud  signature  used  in  this  test  is  small,  the  FLS  test  can  be  severely 
affected  by  sensor  noise  in  the  chaimel  3  sensor  data  which  often  exceeds  the  cloud 
threshold.  The  result  of  the  noise  is  misclassification  of  clear  data  as  cloudy.  The  results 
of  the  FLS  test  are  submitted  to  a  noise  filter  in  the  TACNEPH  algorithm  which  effectively 
reduces  analysis  error  due  to  the  noise.  The  filter  is  explained  in  detail  in  Section  5.2.3. 

The  HIGH  test  also  uses  channel  3  data  along  with  the  two  long  wave  channels  and 
is  particularly  sensitive  to  optically  thin  cirrus.  In  this  case  the  cloud  signal  is  caused  by 
mixed  surfaces  of  different  temperatures  in  the  sensor  field  of  view.  Due  to  the  non 
hnearity  of  the  Planck  function  over  the  channel  3  bandpass,  warm  radiating  surfaces  have 
a  proportionally  greater  influence  on  the  derived  channel  brightness  temperature  than  do 
cooler  surfaces.  This  effect  is  also  present  at  the  longer  wavelength  channels  but  less 
pronounced.  As  a  result  thin  cirrus  generally  appears  warmer  at  night  in  channel  3  than  at 
either  chaimel  4  or  5.  For  detection  of  cirrus,  channel  5  is  preferred  for  comparison  with 
channel  3  since,  as  discussed  earlier  in  the  CIRRUS  test,  channel  5  temperatures  are 
somewhat  lower  than  channel  4  for  ice  clouds,  thereby  enhancing  the  cloud  signature. 
However,  as  was  also  described  in  the  CIRRUS  test  section,  atmospheric  water  vapor  in 
the  field  of  view  will  also  reduce  the  channel  5  temperature,  potentidly  causing  a  false 
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cloud  signature.  To  test  for  this,  the  same  technique  used  in  the  RATIO  test  to  check  for 
potentially  increased  low  level  water  vapor  is  used  here;  if  the  adjusted  surface  temperature 
climatology  exceeds  a  cutoff  level  then  the  channel  4  data  are  used  in  preference  to 
channel  5.  Thus,  the  HIGH  test  checks  for: 

•  T3  -  T5  >  4.0  K  (cloud  clearing  2.7  K) 

if  the  predicted  dear-scene  temperature  is  less  than  or  equal  to  295  K,  otherwise  the  test  is: 

•  T3  -  T4  >  4.0  (cloud  clearing  2.7  K). 

If  the  HIGH  test  detects  cloud,  bit  6  in  the  intermediate  output  word  is  set.  Since  the  cloud 
signature  is  stronger  for  the  HIGH  test  than  for  the  FLS  test,  no  noise  filtering  of  the 
output  file  is  required;  however,  if  in  the  future  the  sensor  noise  increases,  filtering  may 
become  necessary. 

5.2.3  Noise  Filter 

A  well  known  problem  with  aU  extant  AVHRR  sensors  is  instrument  noise  in 
channel  3.  Noise  affects  the  T3  data  differently  for  each  satellite  and  also  changes  with 
time.  As  such,  filtering  the  sensor  data  to  remove  noise  effects  is  problematic. 
Unfortunately  the  sensor  noise  can  impact  the  accuracy  of  cloud  tests  that  use  channel  3 
data,  particularly  at  night  when  T3  cloud  signatures  are  weakest.  Channel  3  noise  has  not 
been  a  significant  problem  during  day  conditions  since  cloud  signatures  tend  to  be  strong 
relative  to  the  magnitude  of  the  noise  (<  3°  K). 

Attempts  to  filter  the  3.7  /an  sensor  data  before  it  was  passed  to  the  cloud  analysis 
algorithm  were  somewhat  successful  in  removing  the  noise  signature  but  had  the  undesir¬ 
able  side  affect  of  smearing  edges  in  the  imagery  (e.g.,  coastlines,  clouds,  etc.).  Smearing 
had  the  same  affect  as  channel  registration  errors  which  in  turn  introduced  new  errors  in  the 
cloud  analysis  (i.e.,  false  cloud  signatures  along  smeared  boundaries).  To  overcome  this 
problem  and  still  minimize  noise  effects  on  algorithm  accuracy,  a  data  filter  is  appUed  to 
synthetic  images  generated  from  the  results  of  each  individual  cloud  analysis  test  adversely 
affected  by  the  sensor  noise.  In  these  images,  sensor  noise  is  generally  manifested  as  a 
misclassification  of  clear  pixels  as  cloud-filled.  In  clear  areas,  the  noise  patterns  are 
interpreted  by  the  cloud  algorithms  as  very  small  clouds  (averaging  1-5  pixels  in  size),  with 
a  characteristic  speckled  pattern  when  displayed  in  image  form.  The  noise  filter  operates  on 
the  spatial  characteristics  of  the  synthetic  images,  relying  on  the  fact  that  clouds  generally 
form  in  clusters.  The  noise  filter  identifies  and  removes  isolated  cloudy  pixels  that  are  not 
part  of  a  cluster  (i.e.,  form  a  speckled  pattern).  Less  frequently,  cloudy  areas  can  be 
similarly  misclassified  as  clear  due  to  the  sensor  noise.  In  these  situations  the  noise  filter 
fills  in  small  clear  spots  in  a  generally  cloudy  area.  Currently,  sensor  noise  levels  are  low 
enough  that  only  the  results  of  the  nighttime  FLS  test  (T4  -  T3)  require  noise  filtering  (the 
cloud  signature  for  this  test  is  relatively  small),  however,  if  noise  levels  increase  to  a  point 
where  other  channel  3  tests  are  affected  the  same  procedure  may  be  apphed. 

The  noise  filter  is  applied  to  each  of  the  individual  cloud  analyses  as  follows. 

Results  from  an  affected  cloud  test  are  placed  in  an  array  that  has  the  same  spatial 
coordinates  (i.e.,  rows  and  columns)  as  the  original  satellite  image.  Each  element  in  the 
array  is  assigned  a  binary  number  representing  the  cloud  test  result  for  one  pixel:  1  = 
cloud,  0  =  clear.  An  n  x  n  window  is  passed  over  the  analysis  array,  moving  one  element 
at  a  time,  and  the  n  x  n  elements  in  the  window  are  summed.  If  the  window  sum  is  less 
than  a  minimum  threshold  value,  the  center  element  is  set  to  0,  indicating  no  cloud  detected. 


47 


If  the  sum  is  greater  than  the  maximum  threshold,  the  center  element  is  set  to  1,  indicating 
cloud.  If  the  box  sum  falls  between  the  thresholds,  the  value  of  the  center  box  is  left 
unchanged.  The  filter  operation  is  always  applied  to  the  original  rather  than  modified  data 
so  that  summation  operation  is  not  affected  by  data  points  within  the  current  window 
location  that  were  previously  changed  by  the  filter.  Figure  7  illustrates  examples  of  the 
possible  window  combinations.  A  sum  less  than  the  minimum  threshold  implies  that  if  the 
algorithm  classified  the  center  element  as  cloud  it  is  probably  anomalous  since  it  is  not  part 
of  a  reasonably  sized  cluster  (see  Figure  7a).  A  window  sum  greater  than  the  maximum 
threshold  indicates  that  the  majority  of  elements  are  cloudy  and  the  center  pixel  is  probably 
cloudy  also  (see  Figure  7b). 

Cloud  edges  are  generally  well  preserved  using  this  filter  method,  as  illustrated  in 
Figures  7c  and  7d.  In  Figure  7c  the  window  lies  at  the  far  edge  of  the  cloud  while  the 
window  covers  more  of  the  cloud  in  Figure  7d.  In  both  cases  the  center  element  would 
remain  unchanged,  thereby  preserving  the  actual  cloud  edge.  Currently,  window  size  is 
defined  as  5  x  5  pixels  for  land  areas  with  a  minimum  threshold  of  8  and  a  maximum 
threshold  of  17.  Over  water  areas  the  window  size  is  defined  as  3  x  3  pixels  with  a 
minimum  threshold  of  3  and  a  maximum  threshold  of  6.  The  smaller  window  is  used  over 
water  backgrounds  due  to  the  higher  occurrence  of  small  cloud  features. 
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Figure  7.  Cloud  test  result  data  filter  examples.  Each  group  of  boxes  represents  the 
cloud  analysis  results  for  one  filter  window.  A  black  box  signifies  cloud 
has  been  detected;  a  white  box  means  clear;  stippled  boxes  have  been 
changed  by  the  noise  filter  from  cloud  to  clear  (a)  or  clear  to  cloud  (b). 

5.2.4  Intermediate  Cloud  Analysis  Result 

The  intermediate  cloud  analysis  provides  a  synopsis  of  the  cloud  and  background 
test  results  for  each  pixel  in  the  input  image.  An  8-bit  value  is  used  to  characterize  the 
results  of  the  cloud  algorithm  according  to  the  breakdown  in  Table  10.  Descriptions  of  the 
individual  background  surface  and  cloud  tests  are  provided  in  Section  5.2. 1  and  Section 
5.2.2  respectively.  Each  8-bit  output  word  represents  results  for  the  pixel  at  the 
corresponding  file  location  (i.e.,  row  and  column)  in  the  input  image.  This  serves  two 
purposes:  1)  it  simplifies  the  database  management  problem  in  that  the  resultant  array  is  the 
same  size  as  the  input  image  and  2)  it  supports  direct  visual  comparison  between  the  results 
and  input  data  when  both  are  displayed  as  raster  images  on  a  suitable  display  device.  Note 
that  the  intermediate  analysis  contains  the  actual  results  of  each  cloud  test;  individual  cloud 
test  results  are  not  checked  against  or  modified  by  the  results  of  the  background  surface 
tests  until  the  final  analysis  step  described  below  in  Section  5.2.5.  So  if  a  visible  channel 
cloud  test  misclassifies  a  clear  pixel  as  cloudy  due  to  problematic  surface  conditions  such  as 
snow  or  sun  glint,  the  cloud  result  is  maintained  in  this  intermediate  analysis.  Evaluation 
of  intermediate  results  proved  extremely  useful  during  algorithm  development  in  that  the 
researcher  could  evaluate  the  accuracy  of  the  individual  cloud  detection  tests  and  judge  how 
much  of  the  scene  was  actually  misclassified  due  to  surface  conditions. 
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Table  10.  Intermediate  Output  Format  Bit  Encoding 


Bit 

Description 

0 

GLINT  (water),  DESERT  (land) 

1 

COAST  (1)  Or  (0)  other 

2 

CIRRUS  test  (solar  independent) 

3 

COLD  test  (solar  independent) 

4 

CLF  (day)  FLS  (night) 

5 

RATIO  (day) 

6 

BRIGHT  (day)  HIGH  (night) 

7 

SNOW  (day) 

5.2.5  Final  Cloud  Analysis  Result 

As  noted  in  Section  5,  results  from  all  tests,  cloud  tests  and  background  tests,  must 
be  interpreted  jointly  to  provide  a  final  cloud  analysis.  The  mles  according  to  which  a  pixel 
is  declared  cloudy  or  clear  are  based  on  results  of  all  the  cloud  and  background  tests.  For 
nighttime  situations,  defined  by  solar  zenith  angles  greater  than  90°,  the  process  is 
straightforward.  If  any  nighttime  test  detects  cloud,  the  pixel  is  classified  as  cloud-filled. 
These  nighttime  tests  are  the  test  for  fog  and  low  stratus,  FLS  and  the  test  for  thin  cirrus 
cloud,  HIGH. 

During  daytime  conditions,  the  process  for  evaluating  individual  cloud  test  results  is 
more  complex.  Several  of  the  cloud  tests  rely  on  reflected  solar  radiation  and  thus  can  be 
confused  by  highly  reflective  terrestrial  backgrounds  such  as  sun  glint,  snow/ice  cover,  and 
desert.  These  problematic  backgrounds  may  degrade  the  accuracy  of  the  AVHRR  cloud 
analysis  algorithm  if  they  are  mistakenly  identified  as  cloud.  To  avoid  this  problem 
individual  test  results  that  classify  pixels  as  cloud-filled  are  not  used  to  generate  the  final 
cloud  product  when  it  is  likely  they  are  erroneous  due  to  problematic  surface  backgrounds. 
The  background  filter  tests  described  in  Section  5.2.1,  and  the  Geography  database 
described  in  Section  2.7,  are  employed  to  filter  these  backgrounds  from  the  final  cloud 
results.  Filtering  is  achieved  by  negating  the  results  for  a  particular  cloud  test  if  an 
appropriate  background  filter  flag  has  also  been  triggered  for  the  pixel  being  evaluated. 
Table  1 1  provides  a  look  up  matrix  of  the  filters  employed  by  each  of  the  seven  individual 
AVHRR  cloud  detection  tests. 

In  addition  to  background  surface  filters,  several  tests  use  cloud  detection 
thresholds  designed  to  be  more  restrictive  over  problematic  background  surfaces.  All 
restrictions  that  must  be  considered  when  analyzing  results  of  the  individual  cloud  tests  to 
produce  a  final  cloud  classification  are  summarized  below.  Included  in  these  descriptions 
are  solar  zenith  angle  requirements  and  conditions  under  which  the  results  of  the  individual 
cloud  tests  are  not  used  (filtered)  due  to  problematic  background  surfaces. 

CLF 

•  Solar  zenith  angle  must  be  less  than  or  equal  to  90° 

•  Desert  and  sun  glint  backgrounds  require  stricter  cloud  detection  thresholds 

•  Test  result  is  not  used  under  the  following  conditions: 

-  Spectral  snow  test  is  positive  (Section  5.2.1) 
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Table  11.  Background  Surface  Filters  for  Cloud  Tests 


Spectral  Test  Filters 

Supporting  Data  ] 

filters 

Sun  Glint 

Snow/Ice 

Desert 

Snow/Ice 

AFGWC 

Geographic 

Desert 

Geographic 

Coast 

CLF 

/ 

RATIO 

/ 

/ 

/ 

/ 

y 

BRIGHT 

✓ 

/ 

/ 

✓ 

/ 

COLD 

/I 

CIRRUS 

/I 

FLS 

HIGH 

1  During  daytime  conditions 


RATIO 

•  Solar  zenith  angle  must  be  less  than  or  equal  to  80° 

•  Cloud  detection  threshold  maintained  as  a  function  of  humidity 

•  Test  result  is  not  used  under  the  following  conditions: 

-  Sun  glint  test  is  positive  (Section  5.2.1) 

-  Spectral  snow  test  is  positive  (Section  5.2.1) 

-  Spectral  desert  test  is  positive  (Section  5.2.1) 

-  Geography  database  indicates  desert  background  (Section  2.1.3) 

-  Geography  database  indicates  background  is  coast  (Section  2.1.3) 


BRIGHT 

•  Solar  zenith  angle  must  be  less  than  or  equal  to  60° 

•  Water  and  land  backgrounds,  identified  by  the  geography  database  (Section 
2.1.3),  use  separate  cloud  detection  thresholds 

•  Test  result  is  not  used  under  the  following  conditions: 

-  Sun  glint  test  is  positive  (Section  5.2.1) 

-  Spectral  desert  test  is  positive  (Section  5.2.1) 

-  Spectral  snow  test  is  positive  (Section  5.2.1) 

-  Geography  database  indicates  desert  background  (Section  2.1.3) 

COLD 

•  Water,  land,  coast,  and  desert,  identified  by  the  geography  database  (Section 
2.1.3),  use  separate  cloud  detection  thresholds 

•  Test  result  is  not  used  under  the  following  conditions: 

-  Spectral  snow  test  is  positive  (Section  5.2.1) 
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CIRRUS 


•  Test  result  is  not  used  under  the  following  conditions: 
-  Spectral  snow  test  is  positive  (Section  5.2.1) 


FLS 

•  Solar  zenith  angle  must  be  greater  than  90° 

•  Desert  backgrounds  identified  by  geography  database  (Section  2.1.3)  require  a 
stricter  cloud  detection  threshold 

HIGH 

•  Solar  zenith  angle  must  be  greater  than  90° 

•  Cloud  detection  test  channel  combination  selected  as  a  function  of  humidity 


5.2.6  AVHRR  Output  Product 

The  output  product  of  the  AVHRR  Cloud  Analysis  Algorithm  is  an  8-bit  quantity 
termed  the  Mask  and  Confidence  Flag  (MCF).  The  MCF  is  a  bit-mapped  quantity  that 
stores  cloud/no  cloud  information  plus  flags  for  missing  data  and  the  confidence  flag.  Bit 
assignments  for  the  AVHRR  MCF  cloud  analysis  algorithm  output  are  provided  in  Table 
12.  One  MCF  is  produced  for  each  pixel  in  the  input  AVHRR  image.  MCF  bit 
assignments  are  made  as  follows: 


Table  12.  AVHRR  Cloud  Analysis  Algorithm  MCF  File  Bit  Assignments 


Bit 

Assignment 

Description 

0 

Cloud  Mask 

ON  =  Cloud-Filled 

OFF  =  Cloud-Free 

1 

Not  Used  By  AVHRR  Algorithm 

2 

Not  Used  By  AVHRR  Algorithm 

3 

Not  Used  By  AVHRR  Algorithm 

4 

Not  Used  By  AVH^  Algorithm 

5 

Data  Dropout 

ON  =  Missing  or  Unreliable  Data 

6 

Confidence 

0  =  Missing  Data;  1  =  Low; 

7 

Flag 

2  =  Middle;  3  =  High 

Cloud  Mask  -  Bit  0 

The  cloud  mask  bit  is  set  to  ON,  indicating  a  cloud-filled  pixel,  if  the  pixel  is 
determined  to  be  cloud-filled  by  the  final  cloud  classification. 


Bitl 

Not  used  in  this  algorithm 

Bit  2 


Not  used  in  this  algorithm 
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Bits 


Not  used  in  this  algorithm 

Bit  4 


Not  used  in  this  algorithm 
Data  Dropout  -  Bit  5 

The  data  dropout  bit  is  set  if  the  data  for  the  pixel  are  either  missing  or  unreliable. 
Confidence  Flag  -  Bits  6  &  7 

The  confidence  flag  bits  are  set  to  indicate  LOW  (1),  MIDDLE  (2),  or  HIGH  (3) 

confidence. 

5.2.7  AVHRR  Cloud-Clearing  Procedure 

The  cloud-clearing  procedure  of  the  AVHRR  algorithm  simply  requires  running  the 
AVHRR  algorithm  with  less  restrictive  cloud  thresholds.  The  intent  is  to  find  all  cloud  in 
the  scene  with  a  bias  toward  overanalysis.  Note  that  the  thresholds  must  be  set  to  insure 
that  some  clear  areas  are  identified  correctly  in  order  that  the  clear  scene  statistics  database 
be  regularly  updated  with  new  information.  Because  of  the  difficulty  in  discriminating 
cloud  from  certain  backgrounds,  if  a  pixel  is  determined  cloudy  by  any  cloud  test, 
irrespective  of  the  background  type  tests,  it  is  not  included  in  the  clear  scene  statistics. 
AVHRR  cloud-clearing  thresholds  are  noted  in  parentheses  ()  next  to  the  test  eqations. 

5.3  Testing  and  Validation  of  Cloud  Algorithms 

Algorithm  testing  and  validation  using  real-satellite  data  was  a  major  component  of 
the  TACNEPH  program.  A  major  issue  related  to  test  and  validation  of  nephanalysis 
algorithms  is  the  lack  of  universally  recognized  ground  truth  data.  For  TACNEPH,  manual 
analysis  of  sensor  data  was  chosen  for  ground  truth  over  other  potential  sources,  such  as 
surface  observations,  for  two  reasons:  1)  our  belief  that  comparisons  between  surface 
based  and  satellite  based  observations  are  problematic  and  2)  the  high  degree  of  accuracy 
that  can  be  obtained  from  careful  analysis  of  sensor  data.  Satellite  and  surface  based 
observing  systems  do  not  view  cloud  in  the  same  way,  this  is  manifested  in  such  things  as: 
differences  in  viewing  geometry  including,  say,  a  surface  observer’s  tendency  to  see  sides 
of  clouds,  the  surface  observer’s  sky  dome  is  unlikely  to  be  representative  of  the  larger 
satellite  observed  scene,  differences  in  cloud  characteristics  and  obscuration  when  viewing 
from  above  and  below,  misregistration  of  sensor  data  relative  to  the  observers  location,  and 
the  lack  of  reliable  surface  observations  over  most  parts  of  the  world.  Problems  associated 
with  comparisons  of  satellite  and  surface  cloud  estimates  were  studied  extensively  by 
Hughes  and  Henderson-Sellers  (1983).  They  point  out  that  while  cloud  distributions  based 
on  surface  based  observations  show  good  agreement  with  distributions  derived  from 
satellite  data,  direct  comparisons  between  a  given  satellite  analysis  and  specific  surface 
observations  are  problematic.  Even  the  use  of  data  from  large  mesoscale  networks  provide 
formidable  obstacles:  such  networks  are  scarce  and  generally  do  not  even  report  cloud 
amount.  The  recent  FIRE  intensive  field  observing  programs  offer  a  potentid  source  for 
detailed,  high  resolution  cloud  data,  however,  the  coverage  area  was  still  relatively  small 
compared  to  satellite  coverage,  the  data  are  not  easily  obtained,  and  the  program 
concentrated  primarily  on  cirrus  cloud. 
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Manual  analysis  of  sensor  data  for  comparison  with  automated  analyses,  while 
incestuous,  has  the  major  advantage  of  concurrency  (i.e.,  same  viewing  geometry,  solar 
illumination,  time  of  day,  and  coverage).  Also,  trained  human  analysts,  with  the  aid 
modem  image  processing  systems,  can  exploit  spectral  information  in  multispectral  data  in 
addition  to  spatial  feature  recognition  that  has  traditionally  been  the  strength  of  manual 
photo  interpretation.  The  interactive  image  enhancement  and  display  functions  described  in 
Section  3.2  were  used  extensively  by  the  manual  analysts  to  assist  in  correctly  classifying 
satellite  imagery,  Surface  observations  (when  available)  were  not  ignored,  however,  they 
were  used  only  for  guidance  by  the  analyst  interpreting  a  scene  and  not  to  replace  the 
satellite  based  analysis.  Surface  observations  were  particularly  useful  to  detect  or  confirm 
the  presence  of  low  clouds,  fog,  and  multiple  layer  systems. 

5.3.1  Algorithm  Testing 

Testing  of  the  TACNEPH  cloud  detection  algorithms  on  real  satellite  data  was  an 
essential  part  of  the  development  process.  Many  advances  and  refinements  to  the  cloud 
detection  capabilities  resulted  from  direct  manu^  analysis  of  numerous  satellite  images. 
Trained  satellite  data  analysts  examined  the  algorithm  results  by  manually  comparing  the 
intermediate  cloud  analysis  to  the  original  satellite  data  by  displaying  the  satellite  data  as  a 
raster  image  with  a  color  coded  cloud  mask  overlay.  In  total,  over  1500  images  were 
analyzed  in  this  way.  These  provided  an  excellent  record  of  recurring  problems  such  as 
over  or  under-analysis  of  cloud,  seasonal  effects  and  sensor  noise  problems. 

Early  in  the  program,  initial  case  studies  were  performed  on  a  limited  AVHRR 
dataset  from  four  regions  representing  extensive  climatic  variation  (Figure  8):  Saudi  Arabia 
data  included  desert  and  warm  ocean  conditions;  Brazil  data  contained  moist  tropical 
regions;  the  Alaska  data  had  snow,  sea  ice  and  intricate  coastlines;  and  the  northeast  US 
data  provided  temperate  climatic  conditions.  However,  the  data  set  spanned  a  maximum  of 
10  days  and  only  daytime  data  were  available.  At  that  the  time,  there  was  no  access  to 
DMSP  data.  Although  the  AVHRR  data  provided  valuable  insight  into  the  effectiveness  of 
the  cloud  detection  scheme  over  different  climatic  and  surface  regimes,  it  quickly  became 
apparent  that  an  extended  time  series  of  data  from  all  satellites  was  crucial  to  the 
development  of  accurate  robust  algorithms.  Application  of  the  algorithms  over  both  diurnal 
and  seasonal  changes  was  important  as  was  the  required  historical  record  of  satellite 
derived  corrections  to  the  stored  climatology  (Section  4.2). 

A  modification  to  the  TACNEPH  contract  resulted  in  the  purchase  of  two  satellite  ground 
stations  which  were  installed  at  the  Phillips  Laboratory  (PL/GPA);  a  NOAA  ground  station 
collected  data  from  NOAA- 1 1  and  NOAA- 12  starting  in  the  spring  of  1992  and  a  DMSP 
ground  station  collected  either  FIO  or  FI  1  data  beginning  in  the  fall  of  1992.  Over  the 
course  of  a  year,  real-time  transmission  data  from  every  pass  in  the  ground  station  line  of 
sight  were  collected,  ingested  into  TDB  and  analyzed  for  cloud.  Four  ROIs  were  selected 
for  testing  puiposes  as  shown  in  Figure  9.  These  regions  were  chosen  to  stress  the 
algorithm  over  the  most  variable  set  of  climatic  and  background  conditions  for  which  data 
were  available.  The  region  centered  over  Hudson's  Bay  encompassed  some  of  the  more 
difficult  conditions  over  which  to  detect  cloud,  including  large  snow  and  ice  fields  over 
land  and  water,  permanent  and  new  sea  ice  formations  and  large,  rapid  changes  in  land  and 
water  surface  temperatures,  especially  during  the  fall  and  spring.  The  northeast  US  region 
provided  data  representative  of  a  mid-latitude  region  with  moderate  climatic  conditions. 
Variations  in  terrain,  small  lakes  and  coastlines  provided  some  difficult  challenges.  The 
southeast  US  region  over  Georgia  and  Florida  included  tropical  data  with  very  high 
humidity.  In  the  ocean  region,  sun  glint  and  low  clouds  were  the  biggest  challenges. 


53 


Figure  8.  Initial  ROIs  for  testing  of  nephanalysis  algorithms. 


Figure  9.  Regions  of  interest  selected  for  nephanalysis  testing  using  DMSP  and 
NOAA  data  from  the  PL/GPA  direct-broadcast  satellite  ground  stations. 

The  large  quantity  of  data  to  be  analyzed,  from  upwards  of  12  passes  per  day, 
resulted  in  the  need  for  image  processing  and  manipulation  software  to  facilitate  data  access 
and  display.  A  set  of  interactive  utilities  was  designed  and  developed  with  which  the 
analyst  can  display  sensor  data  images  from  any  satellite  and  overlay  intermediate  cloud 
results  as  a  color-coded  cloud  mask  on  a  32-bit  image  and  retrieve  a  variety  of  information 
about  the  sensor  data  and  cloud  analysis  (see  Section  3.2).  This  capability  has  proven  to  be 
an  extremely  powerful  tool  for  analyzing  algorithm  performance  over  different  scenes  from 
the  case  study  data.  Image  information  capabilities  include  pixel-by-pixel  retrieval  of 
brightness  temperature  and  albedo  equivalents  of  image  gray  shades  for  all  sensor 
channels,  latitude  and  longitude,  solar  geometry  including  solar  zenith,  satellite  zenith  and 
solar/satellite  azimuth  angles,  climatology  correction  information  including  reference 
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temperature,  2a  limits,  and  correction  factors,  and  specific  cloud  test  information  including 
channel  differences  and  ratios.  Display  software  also  provides  for  application  of  various 
normalization  functions  to  the  displayed  imagery.  The  routine  contains  a  list  of 
mathematical  functions,  such  as  solar  and  bi-directional  reflectance  distribution  function 
(BRDF)  correction  factors,  which  are  specified  by  a  coded  input  parameter.  The  selected 
mathematical  operation  is  then  automatically  applied  to  the  source  data  loaded  in  display 
memory  and  results  are  loaded  and  displayed  in  a  separate  area  of  memory. 

Daily  algorithm  testing  was  instrumental  in  highlighting  both  the  strengths  and 
weaknesses  of  the  cloud  algorithms.  The  findings  led  to  major  changes  in  the  algorithms, 
including  development  of  new  tests  as  well  as  modifications  to  existing  tests.  Examples  of 
algorithm  characteristics,  strengths,  and  weakness  that  were  identified  and  addressed 
during  the  testing  process  are  provided  here  to  give  the  reader  a  sense  of  how  the 
algorithms  perform  over  different  conditions. 

Probably  the  biggest  challenge  for  the  algorithms  is  accurate  discrimination  of  low 
cloud  over  highly  reflective  clear  backgrounds,  specifically  snow  and  ice,  sun  glint  and 
desert,  using  the  daytime  AVHRR  tests,  BRIGHT,  RATIO,  and  (XF  and  the  OLS  visible 
and  bispectral  tests.  Lx)w  cloud  is  a  difficult  problem  because  it  is  thermally  indistinct  from 
the  surface  and  therefore  not  readily  detected  by  the  infrared  cloud  tests.  As  detailed  in 
Section  5.2.1,  background  surface  tests  in  the  AVHRR  algorithm  were  used  to  identify 
potentially  problematic  backgrounds.  The  discrimination  of  snow  and  ice  from  cloud  using 
SNOW,  is  good  when  there  is  sufficient  solar  illumination  (i.e.,  solar  zenith  angle  of  less 
than  60°).  Between  60  and  80  degrees  of  solar  zenith,  the  detection  of  snow  is  uncertain. 
Although  the  SNOW  test  does  not  detect  all  the  snow  in  a  scene  due  to  weak  spectral 
signatures,  the  only  condition  where  the  algorithm  over  analyzes  snow  is  within  cloud-on- 
cloud  shadows.  The  main  disadvantage  of  SNOW  is  that  it  is  not  applicable  when  the  solar 
zenith  angle  is  greater  than  80®,  so  at  night  there  is  no  way  to  classify  snow  backgrounds. 
Anytime  existing  snow  backgrounds  are  not  detected,  the  potential  for  problems  arise.  All 
cloud  tests  have,  at  one  time  or  another,  misclassified  cloud-free  snow  surfaces  as  cloud 
during  testing.  The  problem  is  most  serious  for  the  AVHRR  COLD  and  CIRRUS  tests  and 
the  OLS  bispectral  algorithm,  which  can  classify  large  snow  fields  as  cloud.  This  problem 
was  seen  repeatedly  over  the  Hudson's  Bay  ROI  from  early  fall  through  late  spring. 
Modifications  to  the  SNOW  test  and  the  cloud  tests  improved  detection  accuracy 
considerably  but  did  not  eliminate  the  problems.  In  the  OLS  algorithm,  there  is  no  means 
of  detecting  snow  s^trally;  the  AVHRR  algorithm  relies  on  3.7  //m  channel  for 
snow/cloud  discrimination,  however,  there  is  no  comparable  band  on  the  OLS  instrument. 

Sun  glint  over  water  surfaces  is  also  a  problem  for  the  AVHRR  and  OLS  tests  that 
rely  on  visible  data.  In  the  AVHRR  algorithm,  the  GLINT  background  filter  test  detects 
enough  sun  glint  to  remove  false  cloud  classifications  from  the  BRIGHT  and  RATIO  tests. 
However,  the  CLF  test  (T3  -  T4)  for  low  cloud  is  sensitive  to  levels  of  glint  too  low  to  be 
detected  by  the  GLINT  test.  To  compensate,  the  CLF  test  is  modified  for  water  points 
whenever  the  solar/satellite  geometry  has  the  potential  to  support  sun  glint.  Reliance  on 
pure  geometric  considerations  is  less  restrictive  than  spectrd  tests  because,  depending  on 
sea  state  (an  unknown  within  TACNEPH),  the  area  of  specular  reflection  can  be  large  and 
oriented  in  whatever  direction  the  wind  is  from.  Since  foere  is  no  way  to  predict  these 
effects,  all  locations  that  could  support  glint  are  assumed  to  be  contaminated.  For  the  CLF 
test,  detection  of  cloud  in  potentid  glint  regions  is  restricted  by  raising  the  cloud  signature 
threshold.  Fortunately  this  does  not  significantly  impact  the  accuracy  of  the  AVH^ 
analysis,  since  other  tests,  in  conjunction  with  the  GLINT  test,  do  provide  good  results. 

The  main  danger  is  avoiding  over  classification  in  these  conditions.  In  the  OLS  algorithm, 
there  is  no  spectral  sun  glint  test  so  that  all  cloud  tests  rely  on  geometric  glint  filters.  For 
this  algorithm,  sun  glint  is  a  more  serious  problem  since  restrictions  on  the  use  of  visible 
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data  directly  impacts  the  ability  to  detect  low  cloud.  For  both  algorithms,  water  points 
incorrectly  classified  as  land  in  the  geography  database  remain  a  problem. 

Cloud  detection  over  reflective  land  surfaces  such  as  desert  posed  an  additional 
problem.  While  the  snow  and  sun  glint  filters  were  part  of  the  original  AVHRR  algorithm, 
the  DESERT  test  was  developed  when  real-data  tests  showed  that  the  geography  database 
desert  classification  did  not  sufficiently  identify  problem  locations.  This  DESERT  test  has 
proven  to  be  extremely  effective  in  identifying  reflective  land  surfaces  that  can  cause  false 
cloud  signatures  in  some  cloud  tests.  Using  DESERT  in  conjunction  with  the  geography 
database  correctly  classifies  almost  all  problem  backgrounds  of  this  type.  Again  due  to  the 
absence  of  a  3.7  /tm  sensor  channel,  the  OLS  algorithm  must  rely  solely  on  the  geography 
desert  classification  resulting  in  situations  where  over-analysis  of  cloud  over  misclassified 
desert  regions  can  occur.  A  proposed  solution  to  the  lack  of  snow  and  desert  tests  in  the 
OLS  algorithm  is  to  use  the  AVHRR  DESERT  and  SNOW  test  results  to  build  dynamic 
desert  and  snow  databases  for  the  local  region. 

One  source  of  error  in  low  cloud  detection  results  from  a  combination  of  the  coarse 
resolution  and  the  minimal  information  content  of  the  geography  database.  Geography 
problems  are  seen  repeatedly  to  affect  many  tests.  For  example,  both  the  OLS  and 
AVHRR  single  channel  infrared  threshold  tests  rely  on  a  comparison  of  predicted  dear- 
column  surface  brighmess  temperature  with  the  satellite  observed  brightness  temperature  to 
detect  cloud.  Within  an  analysis  region,  the  satellite  temperature  of  the  warmest  pixel  is 
used  to  estimate  the  clear  colunon  temperature  (Section  4.2).  When  the  clear  region 
contains  a  large  temperature  gradient,  such  as  often  occurs  between  land  and  water,  the 
warmer  surface  is  declared  clear  while  the  colder  surface  can  be  misclassified  as  cloudy. 
This  problem  can  be  rectified  by  segregating  the  land  and  water  points  and  calculating 
separate  predicted  dear-scene  temperatures  for  each  group.  However,  in  TACNEPH,  the 
resolution  of  the  available  geography  database  (10')  is  much  coarser  than  the  satellite  data 
(1  km).  Coastlines  are  poorly  defined  and  rivers  and  small  lakes  are  generally  not 
identified.  Accurate  division  of  points  by  background  type  is  impossible  thus  perpetuating 
the  problem  of  over-analysis  near  coastlines.  The  problem  is  made  worse  by  the  fact  that 
the  geography  database  contains  only  a  small  num^r  of  surface  classification  types  making 
stratification  by  background  type  difficult.  An  example,  in  addition  to  coastlines,  where 
rapid  changes  in  background  type  can  cause  undetected  thermal  gradients  are  high  mountain 
terrain,  such  as  the  Himalayas,  that  are  next  to  low,  flat  lands.  Without  some  indication  in 
the  geography  database  of  their  existence,  the  changes  in  visible  and  infrared  characteristics 
between  these  surfaces  that  can  affect  cloud  algorithm  performance  will  go  undetected. 

It  is  also  recognized  that  both  the  geography  and  surface  temperature  climatology 
databases  are  biased  towards  land  classifications.  The  convention  in  these  databases  for 
mixed  land/water  locations  is  to  use  land  characteristics.  Anytime  this  type  of 
misclassification  occurs  where  there  is  a  contrast  between  either  the  visible  or  infixed 
characteristics  of  the  backgrounds,  the  possibility  for  degradation  of  cloud  algorithm 
performance  exists.  Examples  include  sun  glint  over  locations  misclassified  as  land,  cold 
water  adjacent  to  warm  land,  and  mixed  fields  of  view  averaging  out  to  a  false  cloud 
signature  in  the  AVHRR  RATIO  test.  In  the  original  cloud  algorithms,  the  RATIO  test 
routinely  identified  mixed  land/water  fields  of  view,  such  as  occur  along  coastlines  and 
rivers,  as  cloud.  The  solution  was  to  negate  positive  results  from  the  RATIO  test  for  points 
identified  as  coastline  by  the  geography  database.  Rivers  and  other  small  bodies  of  water 
too  small  to  be  included  in  the  geography  remain  a  problem. 

Areas  of  high  humidity  are  another  problem  that  affects  the  AVHRR  RATIO  and 
HIGH  tests.  During  testing  this  was  seen  most  often  in  the  southeast  US  ROI  (Figure  9). 

In  the  daytime,  high  concentrations  of  suspended  aerosols  near  the  surface  cause  increased 
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scattering  in  the  visible  AVHRR  channel  1  relative  to  channel  2.  Over  vegetated  land  this 
can  produce  a  channel  2/channel  1  ratio  that  is  typical  of  cloud  rather  than  clear 
background.  At  night  in  humid  regions,  the  HIGH  test  can  detect  more  sub-visual  cirrus 
than  can  be  found  in  the  manual  analysis.  The  aerosols  attenuate  the  channel  5  radiance 
translating  to  a  lower  derived  brighmess  temperature.  Under  extreme  cases  the  resulting  T4 
-  T5  difference  can  be  large  enough  to  exceed  the  cloud  threshold.  Since  no  upper  air 
moisture  information  is  available  to  TACNEPH,  the  potential  for  high  humidity  levels  is 
assessed  through  the  predicted  dear-scene  brighmess  temperature.  If  the  predicted 
temperature  is  high  then,  following  the  assumption  that  increased  capacity  for  moisture  will 
result  in  increase  aerosols,  the  cloud  test  thresholds  are  modified  to  reduce  the  incidence  of 
false  cloud  detection.  This  method  of  parameterizing  humidity  levels  could  likely  be 
improved  with  better  humidity  information  such  as  can  potentially  be  provided  from  the 
SSM/T  and  TOYS  instmments. 

Another  problem  for  some  cloud  tests  are  multiple,  vertically  stacked  cloud  layers. 
While  basic  cloud  detection  is  seldom  a  problem,  AVKKR  tests  sensitive  to  thin  cirrus  can 
be  confused  when  the  ice  cloud  occurs  over  an  optically  thick  water  cloud.  Since  these 
AVHRR  tests  have  potential  for  discriminating  thin  cirrus  from  other  types  of  cloud,  it  is 
important  to  be  aware  of  this  condition  for  applications  that  require  information  on  optically 
thin  cirrus. 


5.3.2  Validation  Procedure 

As  discussed  above,  the  TACNEPH  cloud  detection  algorithms  were  routinely  and 
extensively  tested  by  visually  comparing  the  intermediate  cloud  analysis  (displayed  as  a 
color  coded  cloud  mask)  to  me  original  satellite  data.  However,  a  separate  objective 
algorithm  validation  procedure  was  developed  to  provide  a  more  rigorous  and  quantitative 
measure  of  algorithm  accuracy.  The  TACNEPH  cloud  algorithm  validation  plan  was 
presented  and  approved  at  me  22-23  May  1990  contract  program  review.  It  called  for 
statistical  comparisons  between  manually  analyzed  sensor  data  and  me  automated  algorithm 
results  for  globally  varying  scenes.  This  approach  was  used  extensively  on  previous 
programs  at  me  Phdllips  Laboratory  to  validate  work  done  in  support  of  me  AFGWC 
RTNEPH  program  and  for  me  SSM/I  calibration/validation  study  (dEntremont  et  al., 

1989;  Gustafson  and  Felde,  1988;  1989).  In  accordance  wim  me  accepted  validation  plan  a 
formalized  procedure  for  generating  manual  analyses  from  multispectral  sensor  data  was 
developed.  This  procedure  is  similar  to  mose  used  in  me  previous  studies  aimough  it  was 
extensively  customized  for  me  TACNEPH  application.  Case  study  data  were  collected  to 
test  me  algorithms  over  a  range  of  conditions  using  available  ground  station  resources. 

TACNEPH  algorithm  validation  consists  of  objective  comparisons  between 
automated  algorithm  results  and  manually  analyzed  satellite  imagery  using  case  studies  for 
four  seasons  over  me  East  central  United  States  and  me  Atlantic  Ocean.  The  output  of  me 
manual  analyses  is  used  as  ground  truth  for  me  purposes  of  mese  comparisons.  This 
approach  was  selected  because  it  was  felt  mat  a  manual  analysis  of  me  available  data  by  an 
experienced  analyst  provided  me  most  accurate  and  consistent  ground  trum  data  possible 
for  evaluating  satellite  nephanalysis  algorithms. 

Case  study  data  were  selected  to  be  representative  of  me  range  of  conditions  me 
algorithms  were  expected  to  encounter  operationally.  It  was  assumed  mat  performance  of 
me  cloud  algorithms  is  dependent  on  (at  least)  cloud  type,  scene  illumination,  and 
background  conditions.  However,  case  study  selection  was  somewhat  restricted  in  mat  the 
only  sources  of  data  readily  available  to  me  TACNEPH  program  were  DMSP/RTD  and 
NOAA/HRPT  direct  broadcast  ground  stations.  Thus  me  geographic  extent  was  Umited  to 
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the  coverage  area  for  these  systems  (i.e.,  the  eastern  US.  and  Canada  to  the  western  North 
Atlantic).  To  exercise  the  algorithms  over  as  broad  a  range  of  conditions  as  possible  given 
the  input  data  constraints  and  available  personnel  resources,  two  ROIs  within  the  coverage 
area  were  selected  to  represent  terrestrial  and  oceanic  backgrounds  (Figure  10).  Data  were 
collected  for  each  ROI  over  8-10  day  periods  from  four  seasons:  June,  September, 
December,  and  March;  for  daytime,  nighttime  and  near  terminator  orbits. 


Figure  10.  Selected  regions  of  interest  for  validation  study;  the  land  ROI  covers  the 
area  35-40  N  latitude,  78-83  W  longitude;  the  water  ROI  covers  the  area 
35-40  N  latitude,  73.5  -  58.5  W  longitude. 

Considerable  emphasis  during  the  validation  process  was  placed  on  the  creation  of 
accurate  and  consistent  manual  cloud  analyses  for  the  selected  case  study  periods.  This 
was  necessary  both  because  of  the  importance  these  analyses  have  as  ground  truth  but  also 
because  this  was  by  far  the  most  time  intensive  part  of  the  procedure.  Processing  of  the 
selected  case  study  data  required  analysis  of  approximately  120  AVHRR  and  60  OLS 
scenes  (differences  in  scene  numbers  result  from  the  PL/GPA  DMSP  ground  station 
limitation  to  receive  transmissions  from  only  one  satellite  at  a  time  due  to  restrictions  on  the 
KG-44a  decryption  device  as  opposed  to  the  NOAA  ground  station  which  can  receive  data 
from  two  satellites  concurrently).  To  assist  in  manually  classifying  and  cataloging  such  a 
large  amount  of  data,  a  comprehensive  interactive  computer  program  called  TBLANK  was 
developed  and  implemented  on  the  dedicated  image  processing  computers  available  at  the 
Phillips  Laboratory  (see  Section  3.2  for  a  description).  TBLANK  provided  both  an 
automatic  interface  to  the  TACNEPH  database  and  a  formalized  methodology  for 
performing  the  analysis  on  whatever  mix  of  visible,  infrared  and  multispectr^  satellite 
imagery  was  available. 

In  addition  to  satellite  data,  the  analyst  also  has  access  to  conventional  surface 
observations  over  the  land  background  region  of  interest  (Figure  10).  Surface  based 
reports  of  cloud  cover  are  used  only  for  guidance  during  the  manual  procedure  and  not  to 
replace  the  satellite  based  analysis.  For  example,  if  an  analyst  suspected  a  fog  or  low 
stratus  deck  from  examination  of,  say,  an  AVHRR  composite  3.7  and  1 1  jim  image,  this 
could  be  confirmed  by  surface  reports  from  that  area.  However,  a  surface  report  of  fog, 
without  supporting  evidence  in  the  satellite  imagery,  would  not  be  extrapolated  to  the  larger 
satellite  scene. 
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The  manual  cloud  analysis  is  considered  as  a  single  blind  procedure  (as  opposed  to 
a  double  blind)  since  the  analyst  has  no  knowledge  of  the  automated  algorithm  results  but  is 
aware  of  the  algorithm  characteristics.  Final  output  products  of  both  the  manual  and 
automated  algorithm  analyses  are  binary  images  depicting  the  cloud-filled  (1)  and  cloud- 
free  (0)  regions  of  the  scene  on  a  pixel-by-pixel  basis.  Fractional  cloud  amount  can  be 
readily  calculated  for  any  selected  grid  size.  For  this  study,  comparison  statistics  were 
computed  over  a  16  x  16  pixel  grid  selected  to  approximate  the  grid  spacing  expected  to  be 
used  in  the  operational  implementation  of  TACNEPH. 

The  TACNEPH  algorithm  validation  study  was  a  combination  of  objective  and 
subjective  analyses.  First,  a  quantitative  or  objective  analysis  was  performed  on  each  case 
study  via  a  statistical  comparison  of  the  fractional  cloud  amounts  determined  by  the 
automated  algorithm  and  the  manual  analysis.  Second,  a  subjective  analysis  was  done 
wherein  the  cloud  detection  results  of  each  case  were  visually  compared.  This  subjective 
analysis  proved  useful  both  as  a  means  of  verifying  the  statistics  and  explaining  anomalous 
results. 


Identification  of  statistics  which  accurately  described  the  results  proved  difficult.  A 
variety  of  statistical  procedures  were  reviewed  including  analysis  of  variance,  intraclass 
correlation  coefficient  and  ordered  rank  statistics.  These  procedures  were  discarded 
because  either  the  cloud  fraction  data  did  not  meet  the  test  requirements  or  the  tests 
provided  little  useful  information.  It  was  decided  that  simple  statistical  measures  (e.g., 
mean  difference  in  cloud  fraction,  absolute  mean  difference  in  cloud  fraction)  in 
combination  with  the  subjective  analysis  provided  the  clearest  understanding  of  the 
validation  study  results. 

5.3.3  Validation  Results 

Comparison  statistics  for  the  study  periods  of  March,  June,  September  and 
December  are  summarized  in  Table  13.  The  statistics  are  stratified  by  surface  type  (land 
and  water)  and  by  orbital  time  (day,  night  and  near  terminator)  to  highlight  any  differences 
in  algorithm  performance  under  varying  scene  illumination  and  background.  A  total  of  120 
AVHRR  and  60  OLS  satellite  images  were  analyzed. 

The  AVHRR  algorithm  exhibits  excellent  agreement  with  the  manual  analysis, 
especially  for  the  June,  September  and  December  time  frames.  The  March  cases  include 
some  anomalous  results  which  will  be  addressed  below.  The  mean  absolute  difference 
(MAD)  in  cloud  fraction  ( E I  Ajacneph  ■  Amanual  • )  was  determined  to  be  the  best 
descriptor  of  algorithm  accuracy  because  it  specifies  Ae  magnitude  of  the  difference  in 
fractional  cloud  amount  irrespective  of  which  analysis  identified  more  cloud.  The  MAD  for 
the  entire  AVHRR  study  averages  1 1  %  with  a  range  of  3%  to  25%.  Excluding  the  March 
land  cases,  the  average  MAD  is  9.5%.  While  the  signed  mean  difference  ( Z  (Ajacneph  - 
Amanual ) )  has  shortcomings  in  that  it  falsely  reduces  the  discrepancies,  it  does  show  that 
there  is  no  systematic  bias  in  the  automated  algorithm  toward  under-  or  over-analysis  of 
cloud.  In  just  half  of  the  categories  in  Table  5-7a  (1 1  out  of  24),  more  cloud  was 
identified  by  the  automated  algorithm  than  by  the  manual  analysis. 

Results  from  scenes  over  land  backgrounds  from  the  March  study  are  significantly 
poorer  than  the  others.  Generally,  the  automated  algorithm  detected  more  cloud  than  the 
manual  analyst.  Inspection  of  the  March  satellite  imagery  and  analysis  showed  that  a  large 
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Table  13.  Statistical  Comparison  of  Fractional  Cloud  Amount  Differences  (%)  for  the 
AVHRR  Algorithm  (a)  and  the  OLS  Algorithm  (b) 


March 

June 

DAY 

Sept 

Dec 

March 

NIGHT 

June  Sept 

Dec 

March 

TERMINATOR 

June  Sept 

Dec 

LAND 

Mean 

Difference 

13.3 

10.6 

0.8 

-2.3 

18.9 

-2.2 

0.9 

-10.2 

23.6 

-8.3 

5.1 

-7.9 

Mean 

Abs  Difference 

14.1 

16.5 

3.48 

3.95 

19.0 

12.6 

14.5 

13.5 

25.4 

10.0 

9.3 

9.1 

WATER 

Mean 

Difference 

3.6 

-8.7 

-8.7 

-6.2 

3.2 

-6.4 

2.3 

-3.2 

7.4 

-2.2 

-1.8 

-11.5 

Mean 

Abs  Difference 

5.6 

11.9 

9.9 

9.7 

5.9 

9.7 

11.1 

13.3 

8.6 

15.9 

8.6 

13.4 

(a) 


March 

June 

DAY 

Sept 

Dec 

March 

NIGHT 

June  Sept 

Dec 

March 

TERMINATOR 
June  Sept 

Dec 

LAND 

Mean 

Difference 

12.9 

16.5 

5.5 

5.4 

-22.7 

-9.7 

-13.4 

-14.6 

-21.3 

4.8 

-8.1 

-7.6 

Mean 

Abs  Difference 

20.9 

20.8 

9.3 

5.7 

22.7 

21.7 

15.4 

18.9 

26.8 

17.4 

8.4 

7.9 

WATER 

Mean 

Difference 

-6.1 

-10.7 

-13.4 

-4.1 

-17,8 

-8.9 

-13.5 

-19,3 

-16.4 

-8.8 

-3.8 

4.9 

Mean 

Abs  Difference 

9.1 

12.7 

17.1 

8.3 

19.0 

9.9 

14.9 

19.9 

17.7 

10.8 

7.9 

12.3 

(b) 


snow  field  was  incorrectly  classified  as  cloud.  During  the  case  study  period,  there  was  a 
blizzard  which  produced  a  lot  of  snow  and  lowered  the  temperature  of  the  leind  surface  by 
up  to  30  K.  The  cloud  algorithm  uses  a  threshold  test  which  relies  on  a  dynamically 
corrected  reference  surface  temperature  that  is  based  on  dear-scene  information  gathered 
from  preceding  days.  The  algorithm  was  unable  to  correct  immediately  for  the  Mastic 
background  temperature  change  and  misclassified  the  anomalously  cold  surface  as  cloud. 
The  analyses  were  poor  for  two  days  which  resulted  in  a  degradation  in  algorithm 
performance  for  March.  However,  once  the  algorithm  adjusted  to  the  new  temperature 
conditions  the  statistics  for  the  remainder  of  the  March  study  were  comparable  to  the  other 
months. 

The  OLS  algorithm  performance  exceeded  expectations  for  the  validation  case  study 
given  the  severe  data  hmitations  of  the  two-channel  OLS  compared  to  the  five-channel 
AVHRR  sensor.  As  described  in  Section  5.1,  the  OLS  analysis  is  most  often  produced 
using  only  the  single  channel  IR  data.  Use  of  visible  data  is  problematic:  at  night  the 
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quality  of  visible  data  obtained  from  the  photomultiplier  tube  (PMT)  is  not  particularly 
suitable  for  cloud  detection,  and  during  dige  day,  the  visible  data  is  often  not  used  because 
of  the  the  difficulty  in  characterizing  the  backgrounds  correctly.  So,  the  accuracy  of  the 
OLS  algorithm  is  critically  dependent  on  the  single  channel  IR  threshold  and  the  ability  to 
produce  a  valid  estimate  of  the  surface  temperature. 

In  spite  of  this  limitation,  the  algorithm  performed  well.  In  16  of  the  24  categories 
shown  in  Table  5.7b,  the  mean  absolute  difference  in  cloud  fraction  (MAD)  between  the 
automated  and  manual  analyses  was  less  than  30%;  the  average  MAD  for  the  entire  case 
study  was  25.2%  with  a  range  of  8.0%  to  49.8%.  Careful  analysis  of  the  individual  cases 
in  each  category  show  that  the  large  MADs  (>  30)  are  usually  the  result  of  one  or  two  very 
poor  analyses  which  skewed  the  data  within  the  category.  For  example,  as  in  the  AVHRR 
study,  the  March  data  was  difficult  to  analyze  correcdy  ^cause  of  snow  contamination. 
Scenes  over  water  back^ounds  proved  unexpectedly  ^fficult  to  classify  especially  during 
the  winter  months.  During  March  and  December,  sunlight  conditions  are  low;  thus  the 
algorithm  relies  more  heavily  on  just  infrared  data.  Thermally  indistinct  clouds  (.e.g.  low 
cloud  and  cloud  edges)  were  often  missed  by  the  algorithm  resulting  in  cloud  amount 
differences  of  between  10  and  20%.  This  problem  was  compounded  by  the  manual 
analyst’s  misclassification  of  Gulf  Stream  eddies  as  low  cloud  decks.  Misclassification 
problems,  either  automated  or  manual,  occurred  less  often  for  land  backrounds.  Why  this 
difference  exists  is  not  well  understood.  It  is  conjectured  that  more  very  low  cloud  existed 
over  water  than  land  or  that  the  reference  climatology  was  not  representative  of  the  water 
temperature.  However,  of  more  concern  than  missed  cloud  edges  is  the  apparent 
systematic  bias  of  the  OLS  algorithm.  In  20  out  of  24  categories,  the  manual  analyst 
detected  more  cloud  than  the  automated  algorithm  (signed  mean  dfference  <  0).  Visual 
examination  and  comparison  of  the  manual  and  automated  analyses  show  that  the 
automated  analysis  is  generally  underanalyzing  low  cloud.  Note  that  for  the  purposes  of 
collecting  comparison  statistics  on  cloud  fraction,  data  that  were  classed  as  partly  cloudy  by 
the  algorithm  were  treated  as  cloud-filled. 

In  this  validation  study  visual  comparison  of  the  algorithm  and  manual  analysis 
results  have  provided  important  insight  into  the  causes  of  disagreements  between  the 
manual  and  automated  analyses.  Interpretation  of  the  results  based  on  comparison  statistics 
alone  is  difficult  largely  because  it  is  not  safe  to  assume  that  the  manual  andyses  are  always 
a  perfect  representation  of  truth.  Situations  have  been  identified  where  the  automated 
analysis  is  superior  to  the  manual  analysis,  yet,  because  of  the  way  the  validation  study 
was  set  up,  these  differences  contribute  to  the  algorithm  error  statistics.  For  example, 
assume  two  theoretical  scenes  A  and  B.  Scene  A  contains  many  small  altocumulus  clouds. 
The  manual  analyst  interpreted  the  cloud  edges  differently  than  the  algorithm  resulting  in  a 
mean  fractional  cloud  amount  difference  of  15%.  Scene  B  is  characterized  by  fog  and  low 
cloud  in  river  and  mountain  valleys.  The  manual  analyst  misses  all  the  fog  and  much  of  the 
low  cloud  while  the  algorithm  identified  it  correctly  resulting  in  a  difference  of  10%.  The 
error  in  scene  A  is  acceptable;  the  error  in  scene  B  is  not  because  entire  cloud  fields  were 
actually  imdetected.  Careful  visual  comparison  of  results  for  each  scene  in  the  validation 
study  have  shown  that  most  of  the  discrepancies  occur  at  cloud  edges.  This  is  chiefly  due 
to  differences  in  the  way  that  a  person  and  the  algorithm  define  edges.  The  analyst  relies 
predominantly  on  textural  information  whereas  the  automated  algorithm  uses  only  spectral 
information.  Cloud  edges  are  often  texturally  or  visually  distinct  from  background  surfaces 
while  for  thinner  clouds  enough  background  radiation  can  pass  through  the  cloud  as  to 
make  it  difficult  to  detect  using  spectral  measures. 
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6 .  Quality  Control  and  Tuning 

Interactive  quality  control  (QC)  techniques  are  required  to  ensure  continued 
accuracy  of  the  TACNEPH  cloud  analyses  and  to  adjust  dgorithm  sensitivity  in  response  to 
mission  specific  requirements.  Contract  requirements  were  to  investigate  techniques  to 
improve  QC  procedures  for  TACNEPH  and  recommend  methods  to  reduce  QC  manpower. 
Work  on  this  task  was  divided  between  development  of  quality  control  flags  and  tuning 
options.  Quality  flags  were  developed  to  provide  an  assessment  of  the  relative  accuracy  or 
reliability  of  a  given  analysis.  Since  the  TACNEPH  algorithm  is  required  to  provide  a 
cloud  analysis  for  all  possible  data  combinations  that  may  be  encountered  in  a  tactical 
terminal,  it  was  recognized  that  the  quality  of  the  analysis  would  vary  with  the  available 
data  mix.  The  requirement  for  data  quality  flags  was  established  to  provide  the  end  user 
with  an  indication  of  the  relative  reliability  of  a  given  analysis.  The  main  objective  of  the 
task  is  to  determine  what  objective  criteria  should  be  used  to  measure  algorithm  accuracy. 
Several  quantifiable  measures  were  considered  including  the  strength  of  the  cloud  signal 
(i.e.,  how  close  to  the  cloud  threshold  is  the  measured  quantity),  the  number  of  tests  that 
separately  detect  cloud,  and  the  suitability  of  the  analysis  for  a  particular  type  of  cloud 
(e.g.,  visible  data  for  cirrus  or  thermal  DR  for  low  stratus). 

Tuning  options  are  needed  to  provide  a  tactical  user  with  a  measure  of  control  over 
the  sensitivity  of  the  cloud  algorithms  while  simultaneously  providing  guidance  on  how  the 
options  are  to  be  applied.  Much  of  the  work  on  this  task  centered  on  how  to  provide  a  user 
who  may  have  little  or  no  knowledge  of  the  cloud  analysis  algorithm,  or  even  meteorologi¬ 
cal  satellite  data  analysis,  with  a  tuning  mechanism  that  would  be  useful  and  easily  under¬ 
stood.  The  approach  is  to  limit  what  the  user  can  do  to  fixed  adjustments  to  modify  the 
algorithm  sensitivity  to  cloud.  The  available  options  are  analogous  to  a  dial  with  three 
settings.  The  center  dial  position  corresponds  to  the  optimal  sensitivity  setting  for  that 
layer,  that  is  the  setting  determined  to  most  accurately  detect  cloud  without  over  analyzing. 
One  position  on  either  side  will  allow  a  relative  increase  in  sensitivity  (with  the  possibility 
with  an  increase  in  false  alarms)  and  on  the  other  a  relative  decrease.  ITiis  should  provide 
the  user  with  sufficient  control  to  adjust  the  algorithm  sensitivity  to  match  a  particular 
application. 

6.1  Analysis  Confidence  Flags 

Data  quality  flags  are  added  to  a  cloud  analysis  to  provide  the  end  user  with  an 
indication  of  the  relative  accuracy  of  the  model  at  a  given  location  under  a  given  set  of 
conditions.  This  was  perceived  as  necessary  since  it  is  known  that  algorithm  accuracy  is 
affected  by  (at  least)  the  mix  of  satellite  data,  background  type,  and  time  of  day.  However, 
it  was  concluded  that  providing  end  users  with  too  much  information  would,  in  most  cases, 
be  counter  productive  and  what  would  likely  be  most  effective  would  be  a  single  straight¬ 
forward  metric.  The  recommendation  was  for  a  three  value  expression  indicating  low, 
moderate,  or  high  confidence  in  the  cloud  analysis  since  this  is  information  that  would 
likely  be  useful  to  tactical  users  and  can  be  easily  extracted  from  the  algorithms.  By  using 
this  ^e  of  categorization  the  quality  measures  for  each  cloud  analysis  can  be  coded  as  a 
2-bit  value.  This  in  turn  simplifies  Ae  presentation  of  the  flag  since  they  can  be  easily 
accommodated  and  displayed  as  synthetic,  color-coded  images  on  a  tactical  terminal. 

An  audit  trail  had  also  been  previously  suggested  as  information  that  would  be 
potentially  useful  to  an  end  user  for  evaluating  cloud  analysis  robustness.  However,  in 
^scussions  at  PL  it  was  decided  that  an  audit  trail  contains  more  specialized  information 
than  most  users  would  want  (or  be  able  to  interpret)  so  it  was  decided  not  to  include  one 
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with  the  data  quality  flag.  Audit  trails  could,  of  course,  be  added  as  a  separate  model 
output  parameter  during  operational  implementation. 

There  was  also  considerable  discussion  about  what  criteria  would  feed  into  a  data 
quality  flag.  Identification  and  quantification  of  conditions  that  affect  algorithm  accuracy 
became  the  major  study  topic  during  the  task,  several  possibilities  were  examined: 

•  strength  of  the  cloud  signal  (i.e.,  distance  from  the  cloud  detection  threshold); 

•  in  the  multispectral  algorithms,  the  number  of  separate  tests  that  detect  cloud; 

•  in  the  single  channel  IR  tests,  the  likelihood  that  the  climatology  correction  factor  is 
cloud  contaminated  (i.e.,  distance  from  the  statistical  2a  limits); 

•  the  relative  skill  of  individual  tests  in  detecting  specific  cloud  types  (e.g.,  relatively 
low  skill  of  single  channel  IR  test  for  low  cloud); 

•  potential  impact  of  water  vapor  effects  on  measured  radiation  (e.g.,  suspected  high 
humidity  areas  at  low  satellite  zenith  angles);  and 

•  potential, sun  glint  regions. 

Data  quality  flags  are  provided  separately  by  the  OLS  and  AVHRR  algorithms. 

6.1.1  OLS  Confidence  Flag  Determination 

OLS  algorithm  accuracy  estimates  are  based  on  pixel  attributes  that  can  be  derived 
from  information  available  to  the  analysis  algorithm  such.  These  include  constraints 
imposed  by  external  factors  and  the  strength  of  the  cloud  signature  as  measured  by  the 
an^ysis  algorithm.  Accuracy  estimates  are  intended  to  provided  the  end  user  with  an 
indication  of  how  much  confidence  to  place  in  the  analysis  for  any  given  pixel  and,  as 
such,  are  referred  to  as  confidence  flags.  Three  levels  of  confidence  are  defined:  LOW, 
MIDDLE,  and  HIGH.  The  eleven  pixel  attributes  listed  in  Tablel4  are  used  to  establish  the 
OLS  analysis  confidence  level. 

Table  14.  Confidence  Flag  Criteria 


Attribute 

Source 

Numeric 

Value 

Sun  Glint  Contamination 

Geometiy  Tests 

-1 

Coast  Background 

GeograpWc  Type  Database 

-1 

Desert  Background 

Geographic  Type  Database 

-1 

Default  Temperature  Correction 

Clear-Scene  Brighmess  Temperature 

-1 

Water  Background 

Geographic  Type  Database 

+1 

Land  Background 

Geographic  Type  Database 

+1 

Visible  and  IR  Channels  Available 

Sensor  Data 

+1 

Cloud  and  Within  15°  K  of  Tdd 

Cloud  Algorithm 

+1 

Cloud  and  Within  10°  K  of  Tdd 

Cloud  Algorithm 

+1 

Cloud  and  Within  5°  K  of  Tdd 

Cloud  Algorithm 

+1 

Cloud  and  Within  3®  K  of  Tdd 

Cloud  Algorithm 

+1 
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A  numeric  value  is  assigned  to  each  identifiable  pixel  attribute  that  affects 
confidence  in  the  analysis  (see  Table  14).  Note  that  pixels  located  over  reflective  or 
variable  background  surfaces  (i.e.,  sun  glint,  coast,  desert)  are  assumed  to  be  more 
difficult  to  analyze  and,  as  such,  are  assigned  a  negative  value.  Similarly,  pixels  located  in 
an  analysis  box  that  require  a  default  temperature  correction  in  the  calculation  of  the 
predicted  dear-scene  brightness  temperature  (Eq.  4-14)  are  considered  to  be  more  suspect 
than  those  that  did  not  use  a  default  correction  and  also  cany  a  negative  value. 

The  numeric  value  is  positive  for  attributes  felt  to  improve  analysis  accuracy.  This 
includes  cases  when  the  cloud  analysis  is  performed  over  a  straight  land  or  water 
background  rather  than  one  of  the  problem  backgrounds  listed  above  and  when  both  visible 
and  IR  channels  are  available  to  the  algorithm  for  analysis.  The  strength  of  the  cloud 
signature,  measured  as  the  departure  of  the  IR  brightness  temperature  or  visible  count  from 
the  respective  cloud  cutoff  value,  is  also  used  as  a  measure  of  confidence  in  the  analysis. 

Confidence  flag  values  for  each  pixel  are  established  by  initially  assigning  a 
numeric  value  associated  with  middle  level  confidence  and  then  adjusting  up  or  down  based 
on  the  attributes  that  apply.  A  final  confidence  value  is  calculated  by  summing  the  numeric 
value  associated  with  all  applicable  attributes.  For  example,  if  the  the  following  attributes 
applied  to  a  given  pixel:  Tjr  =  256,  Tdd  =  268,  both  visible-  and  IR-channel  data  are 
availalble,  and  the  background  surface  is  land;  then  the  OLS  algorithm  would  classify  it  as 
cloud  and  compute  a  confidence  level  as  follows: 


Initial  confidence  level  of  MIDDLE  5 

Land  background  +1 

Visible  and  IR  channels  available  -i- 1 

Within  15°  of  IR  cloud  threshold  +1 

Within  10°  of  IR  cloud  threshold  +1 

Within  5°  of  IR  cloud  threshold  +1 

Total  Value  10 


Conversion  to  a  confidence  flag  value  of  LOW,  MIDDLE,  or  HIGH  is  performed  by 
subjecting  the  numeric  value  to  the  thresholds  defined  in  Table  15.  Thus,  for  the  above 
example,  the  confidence  flag  assigned  to  the  pixel  has  a  value  of  HIGH  since  the  total  of  10 
is  greater  than  or  equal  to  the  HIGH  confidence  threshold  of  9, 

Table  15.  DMSP  Confidence  Flag  Assignment 


Confidence  Level  Value 

Confidence  Flag 

0  <  Value  <  5 

LOW 

6  <  Value  <  8 

MIDDLE 

9  <  Value 

HIGH 

6.1.2  AVHRR  ConHdence  Flag  Determination 

The  AVHRR  algorithm  also  produces  information  on  the  expected  relative  accuracy 
of  the  cloud  analysis  for  each  pixel.  The  level  of  confidence  assigned  to  each  pixel  is  based 
on  the  strength  of  the  cloud  signature  relative  to  the  cloud  threshold  level  for  each  cloud 
test.  Signature  strength  is  defined  in  terms  of  quanta  where  a  quanta  is  based  on  the 
magnitude  of  the  cloud  threshold  associated  with  each  test.  A  numeric  value  representing 
the  relative  strength  of  the  cloud  signature  in  quanta  is  calculated  for  each  test  based  on  the 
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magnitude  of  the  cloud  signal.  Table  16  provides  the  quanta  assignments  for  each  cloud 
test.  The  convention  used  is  that  positive  numbers  indicate  cloud-filled  pixels  and  negative 
numbers  indicate  cloud-free  pixels. 

Quanta  size  are  uniquely  defined  as  fixed  values  for  each  cloud  test  with  the  excep¬ 
tion  of  the  Cirrus  Cloud  Test.  The  quanta  size  for  the  Cirrus  Cloud  Test  is  maintained  as  a 
function  of  the  cirrus  cloud  threshold  interpolated  from  tabulated  values  in  Saunders  and 
Kriebel  (1988).  The  quanta  size  (n)  for  the  Cirrus  Cloud  Test  is  calculated  as: 

n  =  Cirrus  Cloud  Threshold  /  2  . 

Thus,  if  the  Cirrus  Cloud  Test  threshold  is  determined  to  be  7.00  then  the  quanta  size 
would  be  set  to  3.5. 

For  example,  if  the  COLD  cloud  test  (Section  3.2.1)  measured  a  Tgfc  -  T4  difference 
of  16.3  K  and  the  cloud  threshold  were  10  K,  then  the  strength  of  cloud  signature  for  that 
test  would  be  2  quanta: 

diff=6.3  (i.e.,  16.3-10) 

and  5.0  <  diff  <  10.0 

therefore  from  Table  6-3:  Quanta  Magnitude  =  2  and  Confidence  Level  =  MIDDLE 

The  procedure  to  assign  a  confidence  flag  to  each  pixel  is  to  use  the  confidence  level 
associated  with  the  test  that  exhibits  the  strongest  spectral  signature.  When  performing  the 
confidence  flag  determination,  cloudy  signatures  always  take  precedence  over  clear 
signatures.  Thus,  if  one  cloud  test  detected  cloud  with  a  quanta  magnitude  of  1  and  another 
test  produced  a  cloud-free  classification  with  a  quanta  magnitude  of  -2,  the  positive  cloud 
result  would  take  precedence  and  the  pixel  would  be  classified  as  cloud-filled  with  LOW 
confidence. 

6.2  Tuning 

Tuning  options  are  required  to  allow  tactical  user  adjustments  to  cloud  algorithm 
accuracy  to  support  specific  mission  requirements.  For  example,  if  a  given  tactical  mission 
were  particularly  sensitive  to  low  cloud  cover,  the  analyst  may  choose  to  increase  algorithm 
sensitivity  to  low  cloud  at  the  risk  of  falsely  classifying  some  clear  areas  as  cloud.  Similar 
to  the  data  quality  flags,  the  decision  was  made  to  limit  user  choices  to  a  few  simple  and 
easily  understood  options.  This  was  in  recognition  of  the  limited  understanding  a  tactical 
user  is  likely  to  have  about  how  the  algorithm  operates.  Tuning  options  adjust  multiple 
cloud  tests  simultaneously  using  predefined  sets  of  coefficients.  Thus,  when  a  user  adjusts 
algorithm  sensitivity  upward  or  downward  from  the  optimal  point  (as  defined  during  the 
algorithm  test  and  evaluation  studies)  through  a  mechanism  analogous  to  a  dial,  the  tuning 
algorithm  makes  the  appropriate  changes  to  all  cloud  thresholds  simultaneously. 

In  the  OLS  algorithm,  tuning  can  logically  occur  in  two  places.  First  is  in  the 
calculation  of  the  predicted  dear-scene  brightness  temperature  (Section  4.2).  The  second  is 
in  the  definition  of  the  clear  and  cloud  thresholds  (Section  5.1). 
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Table  16.  AVHRR  Quanta  Value  Classification  Assignments 


Cloud  Test  Name 

Quanta 

Size 

Spectral  Signature  Departure  Quanta 
From  Threshold  (diff)  Magnitude 

Confidence 

Level 

COLD 

5.0 

0  <  diff  <  5.0 

+1 

LOW/Cloud-Filled 

5.0  <  diff  ^  10.0 

+2 

MIDDLE/Cloud-Filled 

diff  >  10.0 

>+2 

HIGH/Cloud-Filled 

0  >  diff  >  -5.0 

-1 

LOW/Cloud-Free 

-5.0  >  diff  >  -10.0 

-2 

MIDDLE/CIoud-Free 

diff  <  -10.0 

<  -2 

HIGH/Cloud-Free 

CIRRUS 

n 

0  <  diff  ^  n 

+1 

LOW/Cloud-Filled 

n  <  diff  ^2n 

+2 

MIDDLE/Cloud-Filled 

n  =  Cirrus  Cloud  Threshold  /  2 

diff  >  2n 

>+2 

HIGH/Cloud-Filled 

0  >  diff  >  -n 

-1 

LOW/Cloud-Free 

-n  >  diff  >  -2n 

-2 

MIDDLE/Cloud-Free 

diff  <  -2n 

<  -2 

HIGH/CIoud-Free 

CLF 

6.0 

0  <  diff  <  6.0 

+  1 

LOW/CIoud-Filled 

6.0  <  diff  ^  12.0 

+2 

MIDDLE/Cloud-Filled 

diff  >  12.0 

>+2 

HIGH/Cloud-Filled 

0  ^  diff  >  -6.0 

-1 

LOW/Cloud-Free 

-6.0  >  diff  s  -12.0 

-2 

MIDDLE/Cloud-Free 

diff  <  -12.0 

<  -2 

HIGH/CIoud-Free 

RATIO 

0.075 

0  <  diff  <  0.075 

+1 

LOW/Cloud-Filled 

0.075  <  diff  ^0.15 

+2 

MIDDLE/Cloud-FiUed 

diff  >  0.15 

>+2 

HIGH/Cloud-Filled 

0  ^  diff  i  -0.075 

-1 

LOW/CIoud-Free 

-0.075  >  diff  ^  -0.15 

-2 

MIDDLE/Cloud-Free 

diff  <  -0.15 

<  -2 

HIGH/CIoud-Free 

BRIGHT 

0.03 

0  <  diff  <  0.03 

+1 

LOW/Cloud-Filled 

0.03  <  diff  ^  0.06 

+2 

MIDDLE/Cloud-Filled 

diff  >  0.06 

>+2 

HIGH/Cloud-Filled 

0  S  diff  >  -0.03 

-1 

LOW/Cloud-Free 

-0.03  >  diff  >  -0.06 

-2 

MIDDLE/Cloud-Free 

diff  <  -0.06 

<  -2 

HIGH/CIoud-Free 

FLS 

0.75  ' 

0  <  diff  <  0.75 

+1 

LOW/Cloud-Filled 

0.75  <  diff  1.50 

+2 

MIDDLE/Cloud-Filled 

diff  >  1.50 

>+2 

HIGH/Cloud-Filled 

0  >  diff  S  -0.75 

-1 

LOW/Cloud-Free 

-0.75  >  diff  >  -1.50 

-2 

MIDDLE/Cloud-Free 

diff  <  -1.50 

<  -2 

HIGH/CIoud-Free 

HIGH 

1.0 

0  <  diff  <  1.0 

+  1 

LOW/Cloud-Filled 

1.0  <  diff  ^2.0 

+2 

MIDDLE/Cloud-Filled 

diff  >  2.0 

>+2 

HIGH/Cloud-Filled 

0  s  diff  >  -1.0 

-1 

LOW/Cloud-Free 

-1.0  >  diff  ^  -2.0 

-2 

MIDDLE/Cloud-Free 

diff  <  -2.0 

<  -2 

HIGH/CIoud-Free 

Accurate  prediction  of  dear-scene  brightness  temperatures  is  predicated  on  accurate 
classification  of  reference  pixels  used  in  Eq.  4-9  as  either  cloud-free  or  cloud  contaminated. 
This  decision  is  in  turn  dependent  on  the  AT  limits  calculated  in  Eqs.  4-10  and  4-1 1  cor¬ 
rectly  describing  the  natural  variability  between  the  surface  temperature  climatology  and  the 
sateUite  observed  IR  brightness  temperature.  Limiting  the  range  between  the  AT  limits 
would  have  the  effect  of  reducing  algorithm  sensitivity  to  low  cloud  (i.e.,  cloud  tempera¬ 
tures  close  to  surface  temperature).  This  is  accomplished  by  adjusting  the  2a  factor  in  Eq. 
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4- 7  to  a  smaller  value.  Thus,  the  tuning  option  would  be  to  replace  the  2a  in  Eq.  4-7  to  a 

variable:  _ 

ATmiiij  =  ATj  —  PqGj  (6-1) 

where  po  is  a  decimal  number  that  can  be  adjusted  up,  to  increase  sensitivity  to  low  cloud, 
or  down,  to  decrease  sensitivity.  Use  of  this  approach  has  the  beneficial  aspect  of 
maintaining  the  influence  of  local  variations  in  surface  temperature  on  the  cloud  detection 
process.  However,  it  may  have  the  undesirable  side  effect  of  contaminating  the  dear-scene 
statistics  with  cloudy  pixels  that  will  generally  degrade  the  algorithm  performance.  A  more 
direct  approach  is  to  adjust  the  thresholds  Odd  and  ocdr  in  the  IR  algorithm  (Eqs.  5-1  and 

5- 2)  and  pdd  and  pdr  in  the  visible  algorithm  (Eqs.  5-5  and  5-6).  Changing  the  magnitude 
of  ocdd  and  Pdd  will  have  an  inverse  effect  on  the  algorithm  sensitivity  to  low  cloud  (i.e., 
raise  the  threshold  to  decrease  sensitivity).  Similarly,  adjusting  Odr  and  Pdr  will  affect  the 
contribution  from  partially  filled  pixels  relative  to  clear  pixels.  Increasing  the  thresholds 
will  decrease  the  number  of  pixels  classified  as  partially  cloud-filled. 

The  AVHRR  algorithm  can  also  support  tuning  of  cloud  detection  tests.  Tuning 
capabilities  allow  the  user  to  either  increase  or  decrease  algorithm  sensitivity  to  low, 
optimal  and  high.  The  interface  can  best  be  thought  of  as  a  dial,  each  with  three  settings: 

-1 , 0,  +1.  In  &e  case  of  the  AVHRR  algorithm,  algorithm  sensitivity  can  be  adjusted 
separately  for  low,  middle,  or  high  cloud.  Thus,  in  the  dial  analogy,  three  independent 
dials  would  exist  for  each  cloud  level;  sensitivity  to  low  cloud  can  be  modified  without 
affecting  algorithm  sensitivity  to  mid  or  high  level  cloud.  Changing  the  dial  settings 
changes  the  cloud  thresholds  in  the  appropriate  tests.  Figure  1 1  provides  a  schematic 
showing  which  tests  are  affected  when  sensitivity  to  one  or  more  cloud  types  is  changed. 

One  relationship  is  used  to  tune  all  thresholds  simultaneously  by  modifying  the 
optimal  threshold  by  a  predefined  increment  as  follows: 

new_threshold  =  (( pA  *  increment)  +  1)  *  old_threshold 

where  pA  =  -1, 0, 1  indicates  whether  sensitivity  to  a  type  of  cloud  is  increased  or 
decreased  and  increment  gives  the  percent  change  in  the  threshold.  Table  17  provides 
default  increment  values  for  each  cloud  test.  The  sign  of  pA  is  used  to  change  the  threshold 
value  in  the  correct  direction  (i.e.,  decreased  sensitivity  generally  requires  a  higher 
threshold),  fri  the  current  TACNEPH  implementation,  the  dial  settings  are  manually-input 
command  line  qualifiers,  /LOW  =  c,  /MID  =  c,  /HIGH  =  c,  where  c  =  +1  or  -1  for 
increased  or  decreased  sensitivity  respectively.  Rather  than  using  default  (optimum) 
values,  the  increment  and  decrement  values  applied  to  the  cloud  test  thresholds  may  also  be 
set  on  the  command  line.  However,  this  option  should  only  be  used  by  persons  with  good 
understanding  of  the  algorithm.  It  was  included  for  testing  purposes  and  to  aid  setting  of 
the  default  threshold  v^ues. 

Table  17.  Default  Threshold  Increments 


TEST 

%  CHANGE 

BRIGHT 

.25 

RATIO 

.2 

CLF 

.2 

FLS 

.7 

HIGH 

.25 

COLD 

.2 

CIRRUS 

.2 
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Figure  11.  Schematic  of  recommended  AVHRR  cloud  algorithm  tuning  option. 
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6.3  Quality  Control  Recommendations 

Quality  Control  guidelines  formulated  at  TACNEPH  program  reviews  specified  that 
manual  interaction  or  bogusing  of  analyzed  cloud  fields  would  not  be  supported.  How¬ 
ever,  that  did  not  exclude  the  possibility  of  performing  some  type  of  operation  outside  of 
TACNEPH  since,  for  example,  the  Mark  IV-B  has  the  capability  to  bogus  cloud  analysis 
files  maintained  on-line  in  the  database.  Recommendations  were  required  for  display 
options  that  support  manual  examination  and  evaluation  of  input  data  and  analysis  products 
for  the  purpose  of  quality  control.  Interactive  QC  of  algorithm  results  has  been  a  feature  of 
the  TAOSfEPH  development  program  since  its  inception  (see  for  example  the  discussion  on 
algorithm  testing  in  Section  5.3.1).  Many  of  these  types  of  operations  can  be  supported  in 
a  tactical  terming  and,  as  such,  are  potentially  useful  for  real-time  QC  and  in-depth  analysis 
of  cloud  products. 

The  most  useful  dia^ostic  product  used  during  TACNEPH  algorithm  development 
were  interactive  displays  of  intermediate  algorithm  products  overlaid  on  satellite  imagery. 
Intermediate  products  represented  pixel-by-pixel  results  of  individual  cloud  tests  from  both 
the  OLS  and  AVHRR  algorithms.  Each  individual  test  result  was  carried  as  a  1-bit  value 
(cloud  or  clear)  that  was  used  to  generate  a  synthetic  image  describing  the  test  results  over 
the  spatial  range  of  the  input  image.  The  BIT_TOGGLE  program  described  in  Section  3.2 
was  used  to  interactively  manipulate  synthetic  algorithm  result  images  and  the  input  satellite 
imagery.  BIT_TOGGLE  allow  an  andyst  to  select  for  display  any  individual  satellite 
channel  image,  or  combination  of  channels,  from  the  satellite  sensor  (i.e.,  OLS  or 
AVHRR).  Single  channel  images  are  displayed  as  gray  shades,  multispectral  images  as 
color  composites.  On  top  of  the  satellite  imagery,  the  algorithm  result  images  were 
displayed  as  color  coded  values  in  a  non-destructive  overlay.  Algorithm  results  were 
combined  into  groups  of  eight  so  that  they  could  be  carried  in  a  single  8-bit  values  in 
accordance  with  the  limitations  of  the  display  hardware.  BIT_TOGGLE  options  allowed 
the  user  to  view  results  fi-om  any  single  test  or  combined  results  using  any  mix  of  the  eight 
possible  tests. 

This  approach  would  transfer  well  to  a  tactical  terminal  since  the  image  display 
capability  is  well  within  the  scope  of  modem  imaging  computers.  All  functions  can  be 
implemented  using  color  look  up  tables.  Generation  of  multispectral  color  composites 
requires  some  preprocessing  to  be  done  using  look  up  tables,  but  even  that  restriction  is 
eliminated  with  24-bit,  full  color  systems.  The  benefits  of  implementing  this  type  of 
capability  are  consistent  with  the  stated  objective  of  improving  QC  procedures  and  reducing 
manpower  requirements  since  it  allows  the  QC  analyst  to  quickly  and  accurately  evaluate 
the  dgorithm  results  through  direct  comparison  with  the  satellite  imagery.  In  the  tactical 
environment  this  approach  has  the  following  desirable  attributes: 

•  allows  knowledgeable  users  to  evaluate  and  interrogate  intermediate  products  as 
guidance  on  second  order  cloud  characteristics  (e.g.,  type,  phase); 

•  provides  visual  clues  on  where  the  algorithm  detected  cloud  using  spectral 
signatures  that  the  human  analyst  might  otherwise  miss; 

•  gives  an  easy  to  understand  and  interpret  representation  of  algorithm  results;  and 

•  supports  fast  QC  of  analysis  accuracy. 
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7. 


ASSIGNMENT  OF  CLOUD  TOP  ALTITUDES 


Required  TACNEPH  cloud  parameters  are  cloud  amount  and  altitude.  As 
discussed  in  Section  1,  these  parameters  are  to  be  generated  using  only  the  resources 
available  on  a  transportable  tactical  terminal.  For  Sis  reason,  the  use  of  real  time  satellite 
sensor  data  is  indicated.  The  previous  sections  have  provided  information  on  the 
TACNEPH  analysis  approaches  for  cloud  amount  from  both  DMSP  OLS  and  TIROS 
AVHRR  cloud  imagers.  Cloud  location  and  amount  are  analyzed  on  a  pixel-by-pixel  basis, 
leaving  it  up  to  the  user  to  decide  how  to  use  this  information.  It  is  assumed  that  the  cloud 
amount  results  will  be  calculated  over  some  type  of  analysis  grid  (e.g.,  GWC  16*  mesh 
polar  stereographic  grid). 

The  requirements  for  cloud  top  altitude  are  driven  both  by  the  need  to  define  cloud 
layer  heights  to  predict  their  movement  and  other  field  requirements.  The  most  recent 
Operational  Requirements  Document  (ORD)  states  that  cloud  layers  are  required  at  high 
vertical  resolution  (300  m)  with  a  horizontal  resolution  at  tacticd  sites  equd  to  that  of  the 
fine  data  cloud  imagery  (i.e.  on  a  pixel  by  pixel  basis).  TACNEPH  was  not  tasked  with 
this  requirement.  Given  the  tactical  constraints  on  the  problem,  a  reasonable  way  to 
establish  cloud  altitude  consistent  with  that  for  cloud  cover  described  in  Section  5  would  be 
to  take  an  average  cloud  top  temperature  over  the  analysis  grid  cell  using  an  appropriate 
infrared  window  channel  (i.e.,  using  only  those  pixels  determined  to  be  cloudy  by  the 
cloud  detection  algorithm)  and  assign  this  temperature  to  an  altitude  based  on  the  use  of  a 
reliable  temperature  profile.  This  altitude  would  be  reported  as  the  cloud  altitude  for  the 
analysis  grid  box.  The  window  channel  average  equivalent  black  body  brightness 
temperature  (EBBT)  for  cloudy  pixels  would  represent  the  temperature  of  the  average  cloud 
top  with  the  analysis  grid  box  when  corrected  for  atmospheric  effects  as  discussed  below. 
(This  algorithm  would  also  be  applicable  to  determine  the  altitude  of  each  cloudy  pixel  in 
which  case  no  averaging  is  required.)  Appropriate  window  channels  include  the  OLS 
thermal  channel  and  channels  4  and  5  of  the  AVHRR  (channel  3  may  be  used  at  night). 

The  simple  algorithm  defined  above  is  summarized  in  Figure  12. 

In  the  context  of  the  cloud  top  altitude  algorithm  defined  here,  this  section  provides 
a  simple  evaluation  of  the  utility  of  SSM/T  derived  temperature  profiles  for  the  assignment 
of  cloud  top  altitude.  This  evaluation  treats  the  original  TACNEPH  task.  Additionally,  we 
provide  guidance  on  tailoring  the  TACNEPH  cloud  top  altitude  determination  based  on 
forecaster  inputs.  We  contend  that  understanding  of  regional  synoptic  situations  is 
essential  to  the  intei^retation  of  TACNEPH  guidance.  Finally  recognizing  the  plarmed 
availability  of  additional  satellite  data  resources  at  tactical  terminals  not  addressed  in  the 
oiigind  TACNEPH  tasking,  we  recommend  improvements  to  the  determination  of  cloud 
top  altitude.  These  include  use  of  SSM/T-2  moisture  data  and  application  of  TOVS 
sounder  data  for  AVHRR  applications. 


7 . 1  Quantitative  Relationships  of  Cloud  Top  Altitude  and  Temperature 
Profile 

As  discussed  in  Section  1,  the  TACNEPH  algorithms  are  designed  to  operate  in  the 
absence  of  any  dynamic  data  source  other  than  direct  satellite  sensor  transmission,  to 
automatically  select  the  optimal  processing  algorithm  in  response  to  data  availability  or 
quality,  and  provide  techniques  for  customization  and  quality  control  of  derived  products. 
The  assignment  of  the  cloud  height  is  based  upon  the  derived  EBBT  of  the  cloud  field  and 
its  association  to  the  satellite  derived  vertical  temperature  profiles.  A  description  of  the 
characteristics  of  the  microwave  temperature  sounder  from  DMSP,  the  SSM/T- 1  and  the 
NOAA  TOVS  can  be  found  in  Falcone  and  Isaacs  (1987)  and  will  not  be  elaborated  here. 
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Figure  12.  Cloud  altitude  algorithm  flow  chart. 

As  discussed  above  the  ORD  states  that  layers  are  required  to  be  distinguishable 
with  a  minimum  vertical  resolution  of  300  m.  In  order  to  ev^uate  the  applicability  of  the 
SSM/T-1  in  cloud  top  altitude  assignment,  we  have  identified  sources  of  error  in  die  deter¬ 
mination  of  cloud  top  altitude  using  the  simple  algorithm  defined  above.  This  provides  a 
means  to  allocate  the  uncertainties  attributable  directly  to  the  temperature  profile  determina¬ 
tion  and  hence  the  SSM/T-1  retrieval.  Sources  of  error  include:  (a)  instrumental  errors 
(due  to  sensor  calibration,  noise  and  quantization  of  the  LWIR  channel  from  which  the 
EBBT  in  Figure  12  above  is  obtained),  (b)  errors  due  to  uncertain  knowledge  of  atmo¬ 
spheric  attenuation  in  the  IR  channel,  and  (c)  errors  in  the  temperature  profile  itself.  The 
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last  source  of  uncertainty  addresses  the  use  of  SSM/T-1  for  temperature  profile  retrieval. 

In  this  context,  we  treat  the  errors  in  the  temperature  profiles  retrieved  from  SSM/T-1 
"system".  This  includes  instrument  and  retrieval  algorithm  sources  of  error  together. 

For  evaluation  purposes  we  translate  temperature  errors  from  each  source  defined 
above  into  equivalent  cloud  altitude  errors  using  a  typical  atmospheric  lapse  rate.  This  is 
based  on  the  assumption  that  cloud  top  altitude  assignment  is  performed  as  shown  in 
Figure  12  associating  an  IR  channel  EBBT  corrected  for  atmospheric  attenuation  with  a 
height  on  a  profile.  Thus,  using  a  typical  value  of  6.8  K/km,  an  error  of  1  K  in 
determining  the  appropriate  EBBT  corresponds  to  a  cloud  top  altitude  error  of  150  m.  It 
should  also  be  noted  that  the  approach  assumes  unit  emissivity  or  so  called  "black"  clouds. 
For  transmissive  clouds  such  as  cirrus  cloud,  measured  IR  radiation  transferred  through  the 
cloud  has  the  effect  of  biasing  the  cloud  top  altitude  to  lower  values,  i.e.  high  cloud  is 
analyzed  as  lower  cloud.  For  the  AVHRR/TOVS  applications  an  approach  to  address  these 
situations  is  suggested  later. 

Instrumental  errors  arise  due  to  sensor  calibration,  noise,  and  to  quantization  of  the 
data.  Sensor  calibration  is  an  issue  in  that  absolute  IR  radiance  values  provide  the  most 
accurate  measure  of  cloud  top  emission.  The  DMSP  OLS  is  designed  to  provide  an  EBBT 
output  scaled  to  the  sensor  response  output,  however  it  is  not  absolutely  calibrated.  The 
AVHRR  is  calibrated.  Overall,  each  1  K  of  drift  in  EBBT  results  in  potential  150  m  shifts 
in  cloud  top  altitude.  This  may  or  may  not  be  an  issue  depending  on  the  importance  of 
relative  or  absolute  measurements  in  the  final  user  application,  hi  the  context  of  the 
analysis  here,  we  judge  sensor  noise  sources  of  error  to  be  small  enough  not  to  affect  cloud 
top  determination.  However,  quantization  of  the  OLS  in  8  bits  over  its  dynamic  range 
introduces  a  granularity  of  about  a  0.5  K.  This  corresponds  to  an  uncertainty  in  cloud  top 
of  about  75  meters.  Thus  no  OLS  cloud  top  determination  can  be  more  accurate  than  this 
number  even  assuming  no  sensor  noise.  (The  AVHRR  uses  a  10  bit  quantization.)  Both 
calibration  and  quantization  are  issues  for  the  OLS  rather  than  the  AVHRR.  For  the  OLS 
an  estimate  of  total  error  due  to  these  sources  might  be  1.5  K  on  average  or  225  m. 

The  source  of  error  due  to  uncertainly  in  water  vapor  profile  in  the  estimation  of 
cloud  top  altitude  based  on  the  single  IR  channel  plus  temperature  profile  approach 
described  above  can  be  understood  in  the  context  of  the  radiative  transfer  equation  (RTE) 
below: 


R„  =  B  (Te)  T  (Zc)  +  /■  B  [T  (Z)]  dt  (Z)  (7-1) 

This  RTE  describes  the  radiance  signal,  Rq,  measured  by  the  IR  channel  for  a  completely 
overcast  pixel.  The  IR  channel  EBBT  is  the  inverse  Planck  function,  i.e.  B'i(T),  of  Ro 
evaluated  at  the  appropriate  average  wavelength  of  the  IR  Channel  used.  The  two 
contributing  terms:  (a)  are  the  Planck  blackbody  radiation  emitted  by  the  cloud  at  the 
temperature  of  the  cloud  top  Tc  (i.e.,  at  cloud  top  altitude  assuming  a  non  transmissive 
cloud)  which  is  attenuated  by  absorption  by  the  atmosphere  between  the  cloud  top  and  the 
top  of  the  atmosphere  along  the  sensor's  line  of  sight  and  (b)  the  emission  by  the 
atmosphere  over  the  same  path.  If  the  atmosphere  were  transparent,  the  transmission  factor 
multiplying  the  Planck  function  would  be  unity,  the  integral  would  be  zero,  and  the 
measured  radiance  would  be  identically  the  Planck  function  of  the  cloud  top  temperature. 
Although  both  the  OLS  thermal  channel  and  channels  4  and  5  of  the  AVHRR  are  so  called 
"window"  channels,  there  are  sources  or  atmospheric  attenuation.  The  predominant 
mechanism  for  attenuation  is  absorption  by  the  water  vapor  continuum.  Additional  effects 
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are  due  to  carbon  dioxide  0. 1-0.2  K,  aerosol  0. 1-0.4  K,  and  transmissive  cirrus. 
Presumably,  much  cirrus  will  be  identified  by  the  cloud  detection  algorithm. 

The  net  effect  of  this  attenuation  is  that  the  black  body  temperature  equivalent  to  Rq 
is  generally  a  few  degrees  cooler  than  that  of  the  actual  cloud  top.  This  effect  can  be 
treated.  Quantitatively,  the  "correction"  for  water  vapor  attenuation,  is  a  function  of 
satellite  look  angle  and  integrated  water  vapor  amount  between  the  sensor  and  cloud  top. 
The  latter  means  that  corrections  are  larger  for  lower  clouds  and  in  regions  of  high 
precipitable  water.  We  have  evaluated  the  difference  between  actual  cloud  top  temperature 
and  cloud  top  temperature  obtained  by  assuming  an  uncorrected  equivalent  black  body 
temperature  using  an  RTE  computer  code.  As  a  worse  case,  the  cloud  top  was  assumed  to 
be  at  the  ground  (i.e.,  maximum  water  vapor  path).  Results  for  a  mid  latitude  atmosphere 
as  a  function  of  integrated  water  vapor  and  sensor  look  angle  are  shown  in  Figure  13. 

Note  that  corrections  of  2-5  K  are  required  at  nadir  and  corrections  of  8  K  may  be  required 
at  large  viewing  angle/water  vapor  paths.  If  uncorrected,  these  temperature  deficits 
correspond  to  incorrect  altitude  assignments  of  500-750  m,  all  systematically  biased  to  put 
cloud  higher  in  altitude. 

This  source  of  uncertainty  can  be  mitigated  by  using  a  climatological  water  vapor 
correction  or  a  collocated  water  vapor  sounding  sensor.  The  required  climatological 
correction  would  be  location  and  seasonally  dependent.  The  error  associated  with  using  a 
climatological  profile  would  depend  on  the  local  variance  of  water  vapor.  For  the  case 
discussed  above  the  error  in  EBBT  using  a  climatological  profile  is  about  1.5  K 
corresponding  to  about  225  m  using  our  assumed  lapse  rate.  This  estimate  is  based  on 
knowledge  of  the  climatological  variance  about  the  mean  profile.  Note  that  for  any  specific 
situation,  the  error  in  using  the  mean  profile  may  be  much  larger.  Note  that  if  the  water 
vapor  retrievals  from  the  SSM/T-2  are  used  there  is  potential  to  reduce  the  corresponding 
error  in  EBBT  by  a  factor  of  about  three,  i.e.  to  less  than  0.5  K. 

The  final  source  of  error  is  that  due  to  the  temperature  profile  retrieval  accuracy 
available  from  the  sounder  collocated  with  each  imager  (SSM^-1  for  the  OLS  and  TOYS 
package  for  the  AVHRR).  Statistical  information  on  the  error  characteristics  of  the  vertical 
temperature  profile  (VTP)  assignment  provided  by  the  satellite  sensors  is  complied  on  an 
ongoing  basis  at  NESDIS.  Figure  14  provides  an  analysis  of  mean  error  and  bias  between 
VTP’s  obtained  from  the  DMSP  SSM/T-l  and  NOAA  TOYS  and  RAOBS  for  various 
height  levels  in  the  atmosphere  (Reale  et  al.,  1992).  These  data  are  for  a  six  day  period  and 
have  been  stratified  into  three  latitudinal  (only  the  N.  hemisphere  extra  tropics  are  shown). 

As  noted  in  Figure  14,  the  characterization  of  temperature  retrieval  error  by  Grody 
et  al.  (1985)  seems  to  hold  true  with  an  RMS  error  of  2.5-3 .0  K  through  the  troposphere 
(coiresponding  to  altitude  assignment  errors  of  up  to  450  m)  and  linear  surface  errors  of 
4.0  K  or  600  m.  These  errors  appear  consistent  for  the  SSM/T  regardless  of  latitude  or 
season.  The  TOYS  accuracy  for  clear  soundings  are  about  2  K.  For  cloudy  soundings, 
results  are  similar  to  the  SSh^-1.  There  are  some  subtle  differences  in  way  the  errors  are 
reported  between  the  SSM/T-l  and  TOYS.  TOYS  statistics  are  stratified  into  clear  and 
cloudy  (this  distinction  is  not  made  for  SSM/T-l).  In  the  SSM/T-l  statistics,  differences 
are  noted  between  the  retrievals  over  the  oceans  versus  those  from  over  land.  This  is 
attributed  to  the  difference  in  the  surface  emissivity  between  the  differing  backgrounds  and 
the  impact  on  the  retrievals.  It  should  be  noted  that  the  combined  effects  of  temperature 
profile  and  water  vapor  uncertainty  increase  for  the  determination  of  cloud  top  altitude  for 
clouds  lower  in  the  atmosphere. 
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Figure  13.  Deficit  between  cloud  top  and  EBBT  as  a  function  of  look  angle  and 
integrated  water  vapor  for  a  mid  latitude  atmosphere. 
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Figure  14,  Vertical  distribution  of  SSM/T-1  temperature  profile  retrieval  errors. 
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The  data  discussed  above  are  routinely  extracted  by  NOAA  and  analyzed  for  use  in 
tailoring  the  VTP  for  use  in  large  scale  global  NWP  models.  Stratification  by  latitude, 
season,  etc.  would  potentially  assist  in  providing  statistics  for  TACNEPH  guidance. 
However,  direct  application  of  the  error  characterization  to  TACNEPH  guidance 
customization  is  a  monumental  task.  The  difficulty  results  from  the  shear  amount  of  data 
available  (NESDIS  runs  error  statistics  on  the  VTP’s  on  a  continuing  basis  year  round)  and 
the  expected  application  of  the  VTP  to  NWP  and  not  to  a  tactical  mission,  per  se. 
Developing  generalities  from  this  small  sample  is  not  recommended.  Further  analysis  of 
larger  stratified  data  sets  is  required  before  reliable  quantitative  evaluations  are  possible. 

Summarizing  the  discussion  of  error  sources  noted  above.  Table  18  gives  typical 
temperature  errors.  Assuming  that  water  vapor  is  corrected  for  using  climatology,  the  error 
in  the  temperature  profile  retrieval  becomes  the  dominant  source  of  uncertainty  in  fhe 
assignment  of  a  cloud  top  temperature  and  hence  cloud  top  altitude.  Based  on  a  using  a 
climatological  water  vapor  correction,  a  typical  lapse  rate  and  assuming  these  errors  are 
uncorrelated  gives  a  total  uncertainly  of  almost  1  km  for  the  OLS  and  0.6  km  for  the 
AVHRR. 


Table  18.  Temperature  Error  Budget  (K)  for  Cloud  Top  Temperature  Assignment 
(multiply  by  0.15  for  cloud  top  altitude  error  in  km) 


Error  Source 

DMSP  (OLS/SSM/T) 

NOAA 

(AVHRR/TOVS) 

Instrument 

1.5 

negligible 

EBBT  Correction 

None 

2-8 

2-8 

Climatology 

1.5 

1.5 

Sounder 

.5 

.5 

Temperature  Retrieval 

Troposphere 

2.5-3.0 

2.0-2.5 

Near  Surface 

4.0 

2.0-3.0 

7.2  Qualitative  Tailoring  of  TACNEPH  Guidance 

The  combined  uncertainty  discussed  above  is  large  enough  to  suggest  that  the 
quality  of  the  TACNEPH  cloud  top  altitude  guidance  be  evaluated  carefully  by  the 
forecaster.  The  duty  forecaster’s  awareness  of  the  synoptic  situation,  is  the  basis  upon 
which  any  tailoring  of  the  cloud  height  assignment  from  TACNEPH  should  be  based.  The 
temporal  continuity  of  the  cloud  fields  as  determined  through  the  local  and  synoptic 
obsen^ation  network  is  a  powerful  tool  in  maintaining  a  credible  association  between  the 
mission  planning  and  execution  forecasts.  Information  garnered  from  other  sources  can 
also  ^sist  in  the  specific  tailoring  of  the  TACNEPH  provided  guidance.  Sources  of 
additional  information  include  local  radiosondes,  observations  by  deployed  units,  pilot 
reports,  inission  debriefings,  and  satellite  interpretation  discussions  available  through 
communication  channels.  Tbe  integration  of  all  or  some  of  these  data  sources  with  the 
TACNEPH  guidance  can  result  in  spatially  and  temporally  consistent  analysis  and 
subsequent  development  of  the  theater  or  battlefield  level  cloud  height  fields.  Another 
important  aspect  is  the  understanding  by  the  forecaster  of  the  specific  needs  of  the  mission 
planning  system  in  terms  of  the  impacts  of  the  cloud  height  assignment.  Weapon  system 
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selection,  loading,  and  deployment  is  usually  contingent  on  operating  within  specified  rules 
of  engagement  and  locally  developed  tactics.  For  example,  if  the  local  situation  requires  a 
weapons  delivery  after  visual  sighting  of  the  target  within  a  specified  height  range,  the 
height  of  the  cloud  fields  becomes  critical  in  determining  the  probability  of  mission  success 
and  the  assignment  of  mission  alternates  should  the  primary  be  unobtainable. 

Another  source  of  potential  information  regarding  the  tailoring  of  the  TACNEPH 
guidance  is  climatology.  We  have  already  mentioned  the  potential  role  of  water  vapor 
profile  climatology  in  correcting  EBBTs  for  water  vapor  attenuation.  There  are  two  aspects 
of  the  climatological  cloud  height,  itself,  pertinent  to  die  use  within  a  tactical  environment. 
First,  the  region^  climatology  of  the  specific  deployment  area  should  provide  general 
spatial  and  temporal  characteristics  of  the  cloud  fields  and  heights.  TMs  information  may 
be  based  on  cloud  type  information  using  empirically  derived  relationships  between  specific 
types  of  cloud  and  expected  vertical  height  placement  and  development.  For  example, 
stratus  cloud  usually  resides  near  the  900  mb  level  but  local  climatology  may  alter  that 
assignment  based  on  local  affects  (i.e.,  topography,  wind  flows,  nearby  water  masses, 
etc.).  Another  pertinent  aspect  of  climatology  for  die  forecaster  to  consider  in  TACNEPH 
tailoring  is  the  use  of  climatological  vertical  temperature  profiles  to  assist  in  the  final  height 
assignment.  Areas  of  expected  cloud  development  may  be  reflected  in  the  climatological 
profiles  which  characterize  seasonal  and  latitudinal  influences  of  moisture  flows.  It  is 
important  to  note,  that  the  TACNEPH  guidance  will  probably  provide  a  spatially  more 
consistent  and  temporally  more  timely  assessment  of  the  cloud  heights  than  climatology 
alone.  However,  the  integration  of  the  TACNEPH  guidance  within  the  context  of  expected 
climatological  behavior  of  the  region  provides  for  increased  opportunities  for  reliable 
mission  forecasts. 

The  verification  of  the  cloud  height  assignment  through  the  vehicle  of  the  mission 
debriefing  and/or  pilot  report  is  helpful  in  developing  local  rules-of-thumb  regarding 
TACNEPH  guidance  tailoring.  It  is  important  to  keep  track  of  the  original  TACNEPH 
guidance  prior  to  any  modification  by  using  climatology  or  synoptic  awareness  so  that  the 
TACNEPH  could  possibly  be  tuned.  In  the  same  sense,  keeping  track  of  whatever 
modifications  which  are  applied  to  TACNEPH  guidance,  can  assist  in  further  refining 
rules-of-thumb  and  determining  the  most  effective  combination  of  TACNEPH  guidance 
and  specific  tailoring. 

7.3  Expected  Improvements  in  VTP/CIoud  Height  Assignment 

Characterization 

The  above  discussion  has  suggested  a  number  of  areas  for  improvement  of  cloud 
top  altitude  assignment  accuracy.  In  this  section  we  summarize  these  as  recommendation 
for  future  study.  First  and  most  obvious  is  the  use  of  collocated  moisture  sounder 
retrievals  to  support  the  determination  of  water  vapor  corrections  for  the  measured  EBBTs. 
In  the  case  of  DMSP,  the  SSM/T-2  data  are  available  for  this  application.  In  the 
calculations  in  Section  7.1  above  we  have  assumed  SSM/T-2  water  vapor  retrieval 
accuracies  like  those  reported  by  the  SSM/T-2  calibration/  validation  efetrt  (Falcone  et  al., 
1992).  Integration  of  the  T-2  moisture  retrievals  for  this  purpose  was  not  tasked  as  part  of 
TACNEPH,  however.  Likewise,  cloud  top  altitude  assignment  for  AVHRR  using  the 
TOYS  temperature  and  moisture  retrievals  was  not  investigated. 

There  are  ongoing  efforts,  particularly  within  the  NOAA/NESDIS  community  to 
improve  the  accuracy,  precision,  and  usefulness  of  the  VTPs  for  operational  NWP  uses. 

In  addition,  efforts  are  also  focusing  on  better  methods  of  cloud  retrieval  using  VTP 
radiance  measurements  (McMillin  and  Zhou,  1994).  The  TACNEPH  guidance  for  the 
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cloud  height  assignment  is  one  of  the  ways  DoD  is  addressing  this  problem.  Other  efforts 
are  involved  in  SERCAA  (Neu  et  al.,  1994)  and  also  in  improving  weather  communication 
within  theater  and  battlefield  environments.  In  particular,  the  cloud  layering  algorithm  from 
SERCAA  should  have  direct  tactical  applications. 

It’s  obvious  that  further  analysis  of  available  error  statistics  of  the  VTPs  could 
assist  greatly  in  providing  better  understanding  and  application  potential  for  the  TACNEPH 
guidance.  In  addition,  better  techniques  for  assigning  IR  EBBT  to  the  vertical  temperature 
level  could  also  assist  in  meeting  this  goal. 

NESDIS,  in  their  effort  to  improve  the  NWP  assimilation  of  the  VTP,  is  developing 
(and  partially  implementing)  an  imagery  based  display  method  of  the  error  characteristics 
(T.  Reale,  personal  communication).  These  imagery  provide  information  on  a  global  basis 
of  the  errors  between  the  VTP  and  co-located  RAOBS.  The  advantage  of  the  imagery 
based  system  is  the  ease  of  viewing  a  global  pattern  and  the  inherent  ^ta  compaction 
involved  in  the  imagery  display  itself.  There  is  potential  for  further  investigating  the 
usefulness  of  these  error  fields  in  characterizing  TACNEPH  guidance  effectiveness.  The 
fields  could  be  provided  as  at  the  time  of  deployment  to  assist  in  the  qualitative  tailoring  of 
the  TACNEPH  guidance  and  then  updated  within  a  certain  timeliness  after  field 
deployment.  Possibilities  may  also  exist  for  extracting  imagery  specific  to  a  deployment 
area  jdter  initial  set  up  is  accomplished.  Modes  of  data  transfer  would  also  have  to  be 
assessed  and  optimized. 

A  temperature  retrieval  process  which  used  a  physical  approach  and  explicitly 
estimated  surface  emissivity  rather  than  using  a  statistic^  value,  would  likely  improve  the 
retrieval  accuracy.  NESDIS  is  in  the  process  of  assessing  the  differences  between 
statistical  methods  (D-matrix)  and  physical  retrievals  but  error  statistics  are  not  readily 
available  as  yet.  In  using  the  TOVS  error  data,  these  retrievals  appear  to  have  RMS  errors 
about  half  of  what  the  SSM/T-1  values  are.  However,  the  comparison  of  TOVS  to 
SSM/T-1  must  be  done  with  consideration  of  the  combined  IR/microwave  character  of  the 
TOVS  versus  the  microwave  only  of  the  SSM/T-1.  Consequently,  the  TOVS  error 
statistics  are  based  on  comparisons  between  cloudy  and  clear  retrievals.  This  probably 
makes  direct  comparisons  of  SSM/T-1  and  TOVS  a  problem  from  a  scientific  approach  but 
probably  irrelevant  to  an  operational  user  who’s  trying  to  use  whatever  data  is  available. 

A  more  direct  comparison  is  available  based  on  the  unified  retrieval  approach 
(Isaacs,  1989;  Moncet  and  Isaacs,  1994)  which  combines  DMSP  5D-2  sensor  data  (OLS, 
SSM/T-1,  SSM/T-2,  and  SSM/I)  in  a  physical  retrieval  combined  with  nephanalysis. 

From  the  discussion  above,  the  advantage  of  this  approach  is  that  it  tackles  the  problem  in 
its  entirety  to  obtain  temperature  and  moisture  proffles,  surface  emissivity,  and  cloud 
information  directly  and  simultaneously.  For  example,  temperature  profile  retrieval 
accuracy  using  5D-2  data  is  up  to  1-1 .5  K  superior  to  statistical  approaches  in  the  lower 
atmosphere  and  moisture  profile  is  obtained  simultaneously.  Additionally,  cloud  liquid 
water  information  is  also  retrieved.  In  the  context  of  cloud  top  altitude  determination,  this 
retrieval  approach  integrates  the  5D-2  sensor  data  to  provide  the  optimal  temperature  and 
water  vapor  retrieval  to  support  correcting  EBBTs  and  assigning  an  altitude.  It  is  worth 
noting  that  the  same  approach  is  applicable  to  the  TO  Vs  package. 

Cloud  top  temperature  determination  is  difficult  for  cirrus  cloud.  The  RTE  defined 
in  7-1  is  not  applicable  to  cimis  cloud  since  it  can  be  transmissive  (i.e.,  its  emissivity  can 
be  less  than  unity).  In  the  case  of  the  NOAA  AVHRR/TOVS  system,  two  approaches  may 
be  used  to  treat  this  problem.  First,  we  can  exploit  the  ability  of  the  TACNEPH  AVHRR 
cloud  detection  algorithm  to  identify  cirrus  cloud  situations.  Once  this  is  accomplished,  a 
correction  factor  can  be  developed  and  applied  to  account  for  cirrus  cloud  emission.  This 
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correction  factor  would  be  in  the  form  of  a  blackbody  temperature  in  degrees  (K)  which 
would  be  subtracted  from  the  measured  IR  EBBT  to  account  for  transmission  of  radiation 
from  below  cloud  top  level.  Essentially  this  would  be  based  on  a  cirms  cloud  microphysi¬ 
cal  properties  used  to  compute  a  cirms  cloud  model  emissivity  for  the  bandpass  of  the 
channel  which  would  then  be  used  to  evaluate  the  required  "correction"  in  an  average  en¬ 
vironment.  An  additional  level  of  sophistication  would  be  to  use  the  magnitude  of  the 
cirms  cloud  signature  in  the  AVHRR  test  to  adjust  the  magnitude  of  the  correction  (larger 
cirms  cloud  signatures  in  the  4-5  test  would  correspond  to  smaller  corrections).  A  second 
and  more  sophisticated  approach  would  be  to  employ  the  CO2  slicing  technique 
(d’Entremont  et  al.,  1992)  to  the  analysis  of  the  TOVs  data.  This  would  provide  the  cloud 
top  pressure  directly. 

Finally,  we  note  that  there  is  considerable  potential  in  applying  modified  forms  of 
the  SERCAA  geostationary  (d'Entremont  et  al.,  1994)  algorithms  to  the  determination  of 
cloud  cover  and  top  altitude  (see,  for  example,  Wylie  and  Menzel,  1989).  Tasking  of 
TACNEPH  did  not  include  treatment  of  geostationary  sensor  data.  This  is  certainly  an 
oversight. 

We  have  summarized  these  recommendations  in  Table  19.  Asa  consequence  of 
these  techniques  and  suggestions  for  the  qualitative  tailoring  of  the  TACNEPH  guidance, 
the  possible  eventual  development  of  mles-of-thumb  as  viable  analysis  and  forecast  aids  is 
indicated. 

Table  19.  Summarized  Recommendations  for  Improved  Retrieval  of  Cloud  Top 
Altitude  in  a  Tactical  Environment 


DMSP 

TTROS 

GED 

Cloud  Cover  and  altitude 

EBBT  "Correction" 

Water  vapor  climatology 

Water  vapor  climatology 

Implement  GEO  algorithms 
Water  vapor  climatology 

Use  SSM/T-2  Retrievals 

Use  TOVS 

Use  GEO  sounder  channel 

Temperature  Retrievals 

Physical  (unified)  retrieval 

Physical  (unified)  retrieval 

data  (if  available) 

Physical  (unified)  retrieval 

Cirrus  Cloud 

AVHRR  test  based  correction 

Imager  based  correction 

CO2  Slicing 

CO2  Slicing 

8 .  Cloud  Thickness  and  Base  From  Multispectral  Cloud 
Imager  Data 

An  algorithm  was  developed  which  utilizes  multispectral  radiance  data  from  cloud 
imagers  such  as  the  AVHRR  to  provide  cloud  optical  thickness  information.  Cloud  optical 
thickness  is  related  to  cloud  physical  thickness  by  ascertaining  cloud  type  and  using  offline 
radiative  transfer  look  up  tables  which  are  dependent  on  cloud  type  microphysical 
properties  such  as  the  cloud  droplet  size  distribution.  Cloud  base  is  determined  from  the 
estimated  cloud  thickness  and  cloud  top  heights  derived  from  thermal  infrared  channel  data. 

Knowledge  of  cloud  base  is  important  for  a  variety  of  defense  tactical  operations  as 
well  as  for  the  determination  of  downward  infrared  flux  required  in  climate  modeling 
applications.  The  current  RTNEPH  approach  for  cloud  base  specification  uses  cloud  type 
dependent  thickness  based  on  climatological  models  and  cloud  height  determined  from 
sensor  data  (Hamill  et  al.,  1992).  Sensor  data  itself,  however,  can  be  exploited  to  provide 
cloud  thickness  information.  TWs  is  accomplished  by  using  cloud  microphysical  models 
based  on  cloud  type  to  define  the  relationship  between  sensor  radiance  values  and  cloud 
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thickness.  Studies  have  investigated  retrieval  of  cloud  properties,  including  cloud  optical 
thickness,  from  passive  multispectral  sensor  data  (King,  1987;  Minnis  et  d.,  1990, 1992; 
Nakajima  and  IGng,  1989;  Twomey  and  Seton,  1980;  Isaacs  et  al.,  1990;  Berendes  et  al., 
1992).  Results  indicate  that  single  channel  sensor  data  depends  on  many  cloud  parameters, 
such  as  phase,  particle  size  distribution  and  cloud  thickness.  For  a  given  cloud  type,  we 
assume  a  specific  particle  size  distribution  which  is  used  to  relate  sensor  incident  radiances 
and  cloud  thickness. 

In  this  section,  we  describe  a  research  grade  algorithm  developed  to  address  the 
task  requirement  to  develop  an  algorithm  which  would  be  an  improvement  over  the  existing 
RTNEPH  "climatology  based"  approach.  In  accomplishing  this  goal,  we  focused  on 
utilizing  the  cloud  optical  thickness  information  potentially  available  from  back  scattered 
solar  radiation.  The  algorithm  was  designed  to  exploit  calibrated  cloud  imager  data  such  as 
that  from  AVHRR  to  “measure”  cloud  diickness.  The  interplay  of  several  algorithms  is 
fundamental  in  the  determination  of  cloud  thickness.  These  algorithms  include  cloud 
cover,  cloud  type,  and  cloud  height.  The  TACNEPH  cloud  base  approach  uses  the 
TACI^PH  AVHRR  cloud  cover  algorithm  (Section  5.2)  and  TACl^PH  derived  cloud 
heights  (Section  7).  Since  there  was  no  explicit  requirement  for  a  TACNEPH  cloud  type 
algorithm,  we  employed  a  straightforward  cloud  typing  scheme  described  below.  The 
main  purpose  of  cloud  type  is  to  select  the  appropriate  cloud  microphysical  model  to  relate 
cloud  optical  and  cloud  physical  thicknesses.  This  is  done  through  off-line  radiative 
transfer  calculations.  These  look  up  tables  (LUTs)  are  cloud  type  dependent. 

Although  cloud  height  is  derived  after  cloud  type,  the  results  of  the  cloud  height 
algorithm  may  be  used  to  improve  on  the  cloud  types,  should  discrepancies  occur  between 
the  computed  cloud  type  and  the  height  assigned  to  it  by  standard  cloud  models. 

8 . 1  Algorithm  Description 

The  algorithm  for  cloud  thickness  and  base  determination  requires:  (a)  identifica¬ 
tion  of  cloudy  regions  using  the  TACNEPH  cloud  cover  analysis;  (b)  typing  of  the  clouds 
to  assign  the  appropriate  radiative  transfer  based  LUT,  and  (c)  calculation  of  cloud  top 
height.  Cloud  top  height  information  is  obtained  by  comparing  corrected  equivalent  black 
body  brightness  temperature  (EBBT)  from  an  infrared  channel  to  a  temperature  profile  for 
the  same  region.  Figure  15  provides  an  illustration  of  the  procedure  used  to  determine 
cloud  thickness  and  base  using  this  approach. 

The  input  data  for  the  algorithm  consists  of  calibrated  cloud  imager  data  (such  as 
AVHRR).  The  requirement  for  calibration  is  based  on  the  need  to  qualitatively  associate 
back  scattered  radiance  in  the  visible  channels  with  cloud  optical  thickness  through  the  off¬ 
line  radiative  transfer  calculations.  The  multispectral  imager  data  is  used  in  three  ways:  (1) 
determination  of  cloud  type;  (2)  determination  of  cloud  optical  thickness,  and  (3)  determin¬ 
ation  of  cloud  height.  The  TACNEPH  innovation  is  the  merging  of  cloud  type  related 
information  (cloud  microphysics),  and  cloud  optical  thickness  (derived  from  visible 
channel  data  using  off  line  radiative  transfer  calculations  based  on  cloud  type)  to  infer  cloud 
physical  thickness  and  the  subsequent  use  of  cloud  height  to  obtain  cloud  base. 

The  functional  flow  of  the  algorithm  proceeds  in  the  following  steps: 

•  The  TACNEPH  AVHRR  cloud  cover  algorithm  is  used  to  delineate  cloud 
covered  regions.  Cloud  thickness  and  base  are  only  inferred  for  cloudy 
regions.  An  description  of  this  algorithm  is  provided  in  Section  5.2. 
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Image  Data  Neural  Net  Spatial  Variance  Table 


Figure  15.  Cloud  thickness  and  base  procedure. 

•  Multispectral  data  are  used  to  infer  cloud  type.  There  are  many  possible  ways 
to  accomplish  this  using  spatial,  spectral,  and  morphological  cloud  features. 
We  have  not  developed  a  standalone  TACNEPH  cloud  type  algorithm,  but 
have  implemented  a  straightforward  hybrid  approach  which  uses  a  neural  net 
to  accomplish  this  step.  This  is  described  in  Section  8.2.  Any  cloud  typing 
scheme  (including  that  available  on  the  Mark  IV-B  tactical  terminal  or  the 
current  RTNEPH  spatial  variance  approach)  can  be  used  (we  used  salient 
aspects  of  this  into  our  approach).  The  cloud  type  output  is  used  to  identify 
one  of  the  off  line  radiative  transfer  LUTs  for  use  in  the  optical  thickness 
determination. 

•  A  visible  channel  (we  have  used  AVHRR  channel  2)  is  used  to  infer  cloud 
optical  thickness.  The  key  to  this  procedure  is  the  use  of  a  look  up  table 
based  on  radiative  transfer  calculations  which  relates  measured  radiance  in  the 
channel  to  the  cloud  optical  thickness.  Sample  look  up  tables  and  the 
procedure  we  used  to  generate  them  are  provided  in  Section  8.3.  Look  up 
tables  are  dependent  on  cloud  type  (from  the  step  above),  satellite  viewing 
angle,  and  solar  zenith  angle.  By  using  the  cloud  type  dependent  look  up 
table,  the  measured  visible  radiance  is  related  to  cloud  optical  thickness  and 
cloud  physical  thickness.  The  latter  relationship  derives  directly  from  the  type 
dependent  microphysical  assumptions  used  in  the  offline  generation  of  the 
LUTs.  Output  of  this  step  is  the  cloud  thickness  in  kilometers  (or  meters)  and 
is  based  on  synthesis  of  the  information  from  the  first  step  and  the  visible 
channel  radiance  data. 

•  The  final  step  is  the  determination  of  cloud  base.  This  is  done  by  using  the 
cloud  top  height  information  provided  by  assigning  a  height  to  the  equivalent 
blackbody  brightness  temperature(EBBT)  from  an  infrared  window  channel 
using  a  collocated  temperature  sounder  profile  or  other  vertical  temperature 
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profUe  (tactically  this  may  be  done  with  a  ground  based  sounder  or  local 
sonde  release).  We  have  sketched  a  procedure  for  cloud  top  height 
determination  in  Section  8.4  using  AVHRR  channel  5  which  allows  for 
correction  of  the  EBBT  using  water  vapor  profile  information  (again  either  a 
satellite  sounder  data  source  or  in  situ  data  may  be  available).  Once  the  cloud 
top  height  is  determined  the  cloud  base  is  obtained  by  subtracting  the  cloud 
thickness  obtained  above  from  the  cloud  top. 


8 . 2  Cloud  Type 

Cloud  type  is  used  to  fix  the  correct  cloud  microphysics  upon  which  the  relation¬ 
ship  between  measured  visible  radiance  and  cloud  thickness  is  determined.  Cloud  type  per 
se  was  not  a  TACNEPH  requirement  and  a  specific  cloud  type  algorithm  was  not 
developed.  To  prototype  the  cloud  base  algorithm  we  implemented  the  following  approach 
to  cloud  type  retrieval.  Any  viable  cloud  typing  scheme  which  is  consistent  with  the  input 
data  may  be  used  to  obtain  cloud  type  for  the  purpose  of  the  TACNEPH  cloud  thickness 
and  base  algorithm. 

The  cloud  typing  portion  of  the  algorithm  uses  both  spectral  and  spatial  information 
applied  to  pixels  identified  as  cloud-filled  to  determine  the  cloud  type  category.  Spectral 
information  alone  is  not  sufficient  to  identify  the  correct  cloud  type  because  clouds  with 
similar  radiative  signatures  may  have  different  morphologies.  The  algorithm  employed 
here  uses  spectral  information  to  separate  the  cloud-filled  pixels  into  four  classes  from 
which,  using  spatial  information,  the  final  cloud  type  is  determined.  The  cloud  typing 
approach  is  illustrated  in  Figure  16.  The  spectral  information  content  of  the  cloud-filled 
pixels  enables  the  grouping  of  clouds  into  four  categories,  which  are: 

•  Cumulus,  Stratus,  Stratocumulus,  Nimbostratus  (Low  Level  Cloud) 

•  Altostratus,  Altocumulus  (Middle  Level  Cloud) 

•  Cirrostratus,  Cirrocumulus,  Cirrus  (High  Level  Cloud) 

•  C!umulonimbus 

This  sub-typing  process  is  performed  by  a  neural  network  that  uses  the  spectral 
content  of  visible  and  infrared  channels,  together  with  background  information  (land  or 
water)  to  discriminate  among  the  four  classes.  For  example,  cumulonimbus  clouds  can  be 
easily  separated  from  the  other  three  classes  due  to  their  high  albedo  values  and  cold 
temperatures.  Although  different  in  a  morphological  sense,  the  cloud  types  in  each  of  the 
four  classes  have  similar  spectral  properties. 

The  neural  network  was  trained  on  a  preliminary  set  of  test  cases  (~30)  taken  fi-om 
four  different  images.  Each  cloud  type  had  a  few  corresponding  test  cases.  The  training 
method  employed  was  back  propagation.  In  the  training  session,  the  network  is  presented 
an  output  corresponding  to  the  given  inputs.  During  training  the  weights  of  the 
connections  between  nodes  in  the  network  are  adjusted  in  such  a  way  as  to  minimi^p:  the 
output  en-or.  Negative  weights  indicate  inhibitory  connections;  positive  weights, 
disinhibitory  ones.  The  inputs  are  normalized  before  being  used  by  the  neural  network  so 
that  albedo  values  and  brightness  temperatures  fall  in  the  same  range  of  values. 

The  network  is  composed  of  an  input  layer  (the  five  channels  and  background 
information),  a  hidden  layer  with  six  nodes,  and  an  output  layer  with  four  nodes  each  for  a 
possible  cloud  type.  There  is  not  a  general  rule  to  determine  the  optimal  number  of 
hidden-layer  nodes,  however,  a  practical  rule  (based  on  direct  experimentation  and  hidden 
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Figure  16.  Cloud  type  calculation. 

nodes  used  by  the  other  implementations)  was  used:  hidden  nodes  =  (the  number  of  input 
nodes  /  2)  +  3.  Increasing  the  number  of  hidden  nodes  does  not  significantly  improve  the 
network;  it  requires  a  longer  training  period. 

The  second  part  of  the  cloud  typing  procedure  uses  spatial  information  to  differenti¬ 
ate  the  cloud  types  within  each  of  the  four  cloud  categories.  More  specifically,  mean  and 
variance  for  n  x  n  pixel  boxes  in  both  a  visible  and  infrared  channel  are  used.  The 
assumption  behind  the  variance  test  is  that  the  higher  the  variance,  the  more  cumuliform  the 
cloud.  Table  20  demonstrates  how  variance  in  a  visible  and  infrared  charmel  is  used  to 
determine  individual  cloud  types  for  low  level  clouds. 

The  output  of  the  algorithm  is  compared  with  the  output  of  the  cloud  height 
algorithm  to  improve  on  the  performance  of  both. 


Table  20.  Variance  Table  for  Low  Level  Clouds 


Visible  Variance 

Infrared  Variance 

Cloud  Type 

Low 

Low 

Stratus 

Low 

High 

Stratocumulus 

High 

Low 

Stratocumulus 

High 

_ High 

Cumulus 
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8.3  Cloud  Thickness 


Knowledge  of  cloud  type  is  used  in  the  determination  of  cloud  thickness.  The 
algorithm  uses  visible  channel  radiance  information  to  determine  cloud  optical  thickness 
and  cloud  thickness  through  the  use  of  cloud  type  dependent  look  up  tables  (LUTs).  These 
LUTs,  which  provide  relationships  between  sensor  channel  radiance  and  cloud  thickness, 
are  built  off  line  using  radiative  transfer  calculations.  Here  we  describe  the  procedure  we 
used  in  building  a  few  prototype  LUTs  for  testing  puiposes.  The  particular  implementation 
used  to  derive  these  tables  is  not  significant.  Any  state-of-the-art  multiple  scattering, 
radiative  transfer  simulation  model  may  be  used. 

Inputs  to  LUT  generation  are  a  model  atmosphere,  an  indication  of  cloud  type, 
thickness  and  height,  surface  properties,  sun/sensor  geometry,  and  sensor  characteristics. 
The  model  atmosphere  defines  such  attributes  as  temperature,  moisture,  and  aerosol 
background  for  latitudinal  zones  such  as  tropical,  mid  latitude,  and  Arctic.  Cloud  type 
information  describes  droplet  size  distribution  and  phase.  Surface  property  information 
defines  the  geography  type,  such  as  land  or  water  background,  as  well  as  its  respective 
albedo  value.  Geometry  information  provides  sun  elevation,  sensor  viewing  angle,  and 
azimuth.  Sensor  characteristics  provide  channel  response  functions  and  dynamic  range. 
Figure  17  illustrates  the  procedure  used  to  generate  LUTs.  In  subsequent  application  of  the 
LUTs  generated  by  this  procedure,  it  is  assumed  that  of  the  above  input  characteristics, 
cloud  type  and  geometry  are  most  important  in  determining  the  relationship  between 
measured  radiance  and  cloud  thickness. 


Figure  17.  Radiative  transfer-based  LUT  generation. 
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Low  spectral  resolution  transmittance  for  clear  atmospheres,  aerosols,  and  the 
surface  is  calculated  using  the  LOWTRAN  radiative  transfer  code  module.  The  MIE 
module  provides  cloud  optical  properties  from  characteristics  based  on  cloud  microphysics. 
Multiple  scattering  radiative  transfer  code  is  provided  by  the  DOM  module  which  employs  a 
discrete  ordinate  method  (DOM)  approach.  As  indicated  in  Figure  17,  all  of  these  radiative 
transfer  code  modules  are  employed  in  LUT  generation  (Lindner  and  Isaacs,  1993). 

Cloud  optical  properties  are  determined  by  their  microphysical  properties  such  as 
droplet  size  distribution  and  phase  (water/ice).  The  key  opticd  properties  used  in  radiative 
transfer  are  the  extinction  coefficient,  single  scattering  albedo,  and  angular  scattering 
function.  These  are  computed  using  Mie  theory  routines  which  require  complex  index  of 
refraction  and  droplet  size  distribution  as  input,  as  indicated  in  Figure  17.  Index  of 
refraction  is  determined  by  phase  while  droplet  size  distribution  is  dependent  upon  cloud 
type.  Droplet  size  distribution  is  defined  as: 

n  (r)  =  Ar«  exp  (-BrT)  (8-1) 

Inputs  to  the  relationship  shown  above  are  provided  in  Table  21. 


Table  21.  Cloud  Type  Dependence 


Cloud  Type 

Density 

(gm'3) 

Mode  Radius 
(l^m) 

A 

a 

B 

Y 

Cirrus 

0.10 

40 

3.6  (-8) 

6 

1.90 

0.5 

Altostratus 

0.15 

10 

6.0  (-4) 

6 

0.60 

1.0 

Low  Lying  Stratus 

0.25 

10 

1.0  (-3) 

6 

0.60 

1.0 

Altocumulus 

0.15 

10 

5.6  (-2) 

6 

3.79 

0.5 

Stratocumulus 

0.25 

10 

9.4  (-2) 

6 

3.79 

0.5 

Fair  Weather  Cumulus 

0.70 

10 

2.6  M) 

6 

3.79 

0.5 

Figure  18  illustrates  a  set  of  typical  AVHRR  Channel  2  LUTs  for  five  cloud  types 
at  a  look  angle  of  0  degrees.  Based  on  this  table  and  determination  of  the  cloud  type,  a 
cloud  thickness  is  determined.  The  effect  of  cloud  liquid  water  content  is  obvious.  For  a 
given  sensor  incident  radiance,  the  higher  liquid  water  content  clouds  are  thinner.  Given 
this  cloud  thickness  and  a  determination  of  cloud  top  height,  base  is  evaluated  as  illustrated 
in  Figure  15. 

8.4  Cloud  Top  Height 

Cloud  top  height  is  obtained  by  comparing  corrected  equivalent  black  body  bright¬ 
ness  temperature  (EBBT)  from  an  infrared  channel  to  a  temperature  profile  for  the  same 
region.  The  brightness  temperature  of  an  IR  channel  (AVHRR  channel  5)  is  adjusted  to 
account  for  the  integrated  water  vapor  in  the  atmosphere  (computed  from  the  vertical 
moisture  profile).  Subsequently,  the  nearest  temperature  profile  is  used  to  obtain  the  height 
and  pressure  of  the  cloud.  Finely,  by  using  information  from  the  cloud  type  algorithm,  the 
height  of  cumulonimbus  and  cirrus  clouds  is  corrected.  In  fact,  cumulonimbus  clouds, 
being  cold,  are  at  times  placed  higher  than  they  are,  and  cirrus  clouds,  for  their  low 
emissivity,  might  be  placed  lower  than  they  are.  The  cloud  height  algorithm  is  shown  in 
Figure  19. 
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Figure  18.  Cloud  thickness  look  up  tables  (LUTs)  for  five  cloud  types. 


Figure  19.  Integrated  cloud  height  calculation. 
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8.5  Results 


The  cloud  thickness  and  base  algorithm  described  in  Section  8.1  has  been  imple¬ 
mented  in  research  grade  code  consisting  of  both  Khoros  and  C  routines  for  testing  and 
evaluation.  The  algorithm  was  applied  to  a  test  case  over  the  North  East  with  both  land  and 
water  background.  A  partly  cloudy  (43%)  area  was  selected  as  the  test  image.  Figure  20 
corresponds  to  AVHRR  channel  2  of  that  image.  The  date  of  the  image  is  Julian  Day  183 
in  the  year  1992.  The  image  is  512x512  pixels,  with  a  resolution  of  approximately  1  km 
per  pixel. 

The  progressive  steps  in  the  algorithm  procedure  as  described  in  Section  8.1  are 
illustrated  by  the  output  images  in  Figures  21  through  25  for  the  test  case  study.  The  first 
step  is  the  determination  of  cloud  cover.  Figure  21  is  the  derived  cloud  mask  of  the  image. 
Cloudy  pixels  are  represented  in  white;  clear  pixels,  in  black.  Figures  21  and  22  can  be 
compared  for  a  qualitative  validation  of  the  cloud  mask. 

Figure  22  contains  the  results  of  the  cloud  typing  algorithm  described  in 
Section  8.3.  These  results  are  derived  using  the  full  daytime  complement  of  five  AVHRR 
channels.  The  cloud  types  that  are  present  in  the  image  are  the  following: 

1.  Cumulonimbus  (black) 

2.  Cumulus 

3.  Stratus 

4.  Stratocumulus 

5.  Altostratus 

6.  Altocumulus 

7.  Cirrostratus 

8.  Cirrus 

9.  Clear  (white) 


A  gray  shade  look  up  table  was  used  to  render  the  different  cloud  types.  Gray 
shade  values  go  up  in  intensity  (transition  from  black  to  white)  from  type  1  (Cb)  to  type  10 
(clear).  We  have  performed  a  manual  cloud  type  analysis  on  this  image  to  determine  the 
validity  of  the  typing  procedure.  Based  on  comparison  of  the  manual  and  objective 
analysis,  there  is  considerable  quantitative  skill  demonstrated  particularly  for  so  called 
compatible  cloud  types  (for  example,  cirrus  and  cirrostratus).  In  particular  it  can  be  seen 
by  comparing  to  Figure  8-6  that  the  very  bright  areas  in  the  original  image  correspond  to 
convective  regions  where  cumulonimbus  clouds  are  identified  by  the  typing  algorithm. 
Notably,  the  ^gorithm  does  well  in  identifying  clouds  with  similar  liquid  water  contents 
which  is  important  to  the  assignment  of  the  appropriate  microphysics  models  used  in  the 
radiative  transfer  based  LUTs. 

Figure  23  contains  the  results  of  the  cloud  height  algorithm  described  in  Section 
8.5.  Cloud  height  is  derived  using  AVHRR  channel  5  EBBTs.  The  range  in  height  is 
approximately  300-9000  meters.  Lower  values  are  rendered  in  white;  higher  values,  in 
black.  A  sanity  check  for  the  cloud  height  determination  is  possible  by  comparing  it  to  the 
original  image  (Figure  20)  and  the  cloud  type  results  (Figure  22).  Note  that  the  highest 
cloud  types  (Cbs)  are  assigned  the  highest  heights,  the  middle  clouds  (grays  in  both 
figures)  the  middle  level  heights,  and  the  lowest  clouds  (see  the  isolated  Cu  and  St  in  the 
central  portion  of  Figure  15)  the  lowest  heights. 
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Figure  20.  AVHRR  channel  2.  Julian  day:  183;  Year:  1992. 


m 


Figure  25.  Cloud  base.  Low  - 100  m  (white);  high  -  8000  m  (black). 
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Figure  24  contains  the  cloud  thickness  values.  These  are  obtained  by  using  the 
cloud  types  in  Figure  22  to  select  appropriate  LUTs  to  determine  cloud  thickness  based  on 
the  approach  described  in  Section  24.  The  range  in  thickness  is  approximately  200-4000 
meters.  Lower  values  are  rendered  in  white;  higher  values,  in  black.  In  general  the  larger 
cloud  thicknesses  in  Figure  24  correspond  to  the  Cbs  and  Cs/Ci  areas.  The  low  and  middle 
clouds  are  determined  not  be  as  thick. 

The  final  step  in  the  algorithm  consists  of  subtracting  the  cloud  thicknesses  in 
Figure  24  from  the  cloud  heights  in  Figure  23.  Figure  25  contains  the  cloud  base  values. 
The  range  in  the  cloud  base  values  is  approximately  100-8000  meters.  Lower  values  are 
rendered  in  white;  higher  values,  in  black.  In  general,  the  results  make  sense  when 
compared  to  cloud  types  in  Figure  22.  Low  clouds  (dark  in  Figure  23;  exception  Cb)  have 
the  lowest  bases  (light  in  Figure  25).  High  clouds  (light  in  Figure  23)  have  the  highest 
bases  (dark  in  Figure  25).  The  mid^e  clouds  have  mid  level  bases  (intermediate  in  both 
figures).  An  exception  to  the  above  are  the  Cbs  which  have  middle  level  bases. 

WhUe  the  above  test  case  results  are  certainly  not  a  definitive  validation  of  the 
algorithm,  they  demonstrate  skill  in  determining  both  cloud  thickness  and  base.  It  should 
be  pointed  out  that  the  current  RTNEPH  approach  for  cloud  base  is  to  assign  a 
climatological  base  to  each  cloud  type.  Based  on  that  approach,  the  cloud  base  results 
would  have  spatial  properties  identical  to  the  cloud  type  result  shown  in  Figure  22.  The 
improved  TACNEPH  approach  illustrated  in  Figure  25  provides  a  much  more  realistic 
spatial  variation  and  dynamic  range  of  cloud  base  by  exploiting  the  information  content  of 
the  visible  channel  radiance  data. 

8.6  Conclusions 

We  have  outlined  an  algorithm  to  determine  cloud  base  using  multispectral  imager 
radiance  data.  In  this  prototype,  the  cloud  type  is  used  to  assign  a  particle  size  distribution 
which  related  physical  cloud  thickness  and  optical  cloud  thicimess  for  a  given  imager 
channel  bandpass.  Off-line  radiative  transfer  calculations  then  support  the  creation  of  look 
up  tables  (LUTs)  to  provide  cloud  thicknesses  per  cloud  type  based  on  sensor  radiance 
data.  As  described  in  the  previous  section,  we  have  applied  a  research  version  of  this 
algorithm  implemented  on  a  workstation  to  a  test  case  and  obtained  reasonable  results. 

TWs  work  was  reported  at  the  1993  CIDOS  meeting  (Isaacs  et  al.,  1993).  However,  no 
extensive  testing  and  validation  of  the  algorithm  has  been  performed.  We  have  made  a 
preliminary  investigation  of  potential  data  sources  for  validation  and  testing  including  the 
use  of  field  ceilometer  data.  This  should  be  pursued  in  the  future  to  obtain  direct  validation 
of  the  procedure  developed  in  this  task. 

9 .  RECOMMENDATIONS  FOR  USE  OF  SURFACE  OBSERVATIONS  OF 

Cloud 

The  use  of  conventional  data  in  TACNEPH  was  suggested  and  requested  primarily 
because  cloud  bases  and  cloud  amounts  in  the  lower  atmosphere  are  often  better  observed 
from  the  ground  than  from  satellite,  especially  when  higher  clouds  obscure  them. 

Although  a  sensible  approach,  it  must  be  recognized  foremost  that  tactical  terminals  in 
general,  and  the  Mark  IV-B  in  particular,  have  no  capability  to  handle  conventional  data. 
Therefore  it  is  not  possible  to  integrate  such  data  into  the  TACNEPH  cloud  analysis  during 
and  even  after  completion  of  the  satellite-only  portion  of  the  analysis.  Thus  the  capabilities 
of  the  Mark  IV-B  must  be  taken  into  account  when  determining  how  best  to  utilize 
conventional  data  as  a  supplement  to  the  TACNEPH  satellite  cloud  analysis  product. 
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Since  surface  observations  of  cloud  are  not  available  on  tactical  terminals,  their  use 
either  during  or  after  satellite  data  processing  is  precluded.  During  TACNEPH 
convention^  data  assimilation  techniques  were  explored  that  depart  from  the  current 
RTNEPH  merge  processor  philosophy  in  which  surface  observations  are  essentially  force- 
fit  into  the  satellite  part  of  the  analysis.  Such  techniques  included  making  run-time 
adjustments  to  the  cloud/no-cloud  thresholds  so  that  the  satellite-observed  cloud  fractions 
agree  with  the  surface-observed  fractions.  Adjustments  would  have  to  be  made  to  account 
for  differences  in  viewing  geometries;  surface  observers  view  clouds  from  below  that  are 
hundreds  of  meters  away,  while  satellite  sensors  observe  clouds  from  above  that  are 
hundreds  of  kilometers  away.  Although  technically  appealing,  such  approaches  were  never 
seriously  considered  for  TACNEPH  because  the  prime  constraint  is  a  lack  of  surface  data 
in  the  tactical  environment. 

Because  of  this  limitation,  it  was  decided  that  any  conventional  data  processing  has 
to  be  limited  to  post-processing  of  the  satellite-only  TACNEPH  analysis  in  an  external 
tactical  weather  facility  that  does  have  access  to  surface  reports  such  as  the  proposed 
Combat  Weather  System  (CWS).  However,  it  is  important  to  point  out  that  conventional 
observations  typically  are  not  available  until  many  minutes  and  even  hours  after  the  satellite 
image  time.  This  even  further  constrains  the  use  of  conventional  data  in  a  timely  sense 
within  the  TACNEPH  analysis  process.  This  essentially  limits  conventional  data 
assimilation  to  at  best  an  RTNEPH  merge  type  of  processing  capability.  Because  the 
"force-fit"  philosophy  of  the  RTNEPH  merge  process  typically  manifests  itself  as  sharp 
discontinuities  in  the  final  RTNEPH  cloud  product,  it  is  recommended  that  surface 
observations  of  cloud  cover  be  used  only  to  interactively  supplement  the  TACNEPH  cloud 
analysis  displays.  If  it  is  decided  that  a  more  invasive  approach  be  taken,  then  TACNEPH 
cloud  analysis  quality  flags  can  be  used  to  provide  information  on  when  and  how  strongly 
to  integrate  surface  observations  of  cloud.  This  represents  a  change  from  the  current 
RTNEPH,  which  replaces  satellite  observations  of  cloud  with  surface  observations 
whenever  they  are  available.  For  example,  spectral  signatures  that  lie  close  to  the  cloud 
detection  thresholds  or  clouds  that  are  only  detected  by  one  cloud  test  have  a  low 
confidence  in  the  TACNEPH  analysis.  Such  points  can  be  considered  for  conventional 
data  assimilation  in  a  more  strongly  weighted  fashion  than  for  points  whose  cloud  detection 
signatures  are  very  strong  (i.e.,  whose  confidence  flags  are  high).  Other  information  that 
is  available  from  Ae  TACI^PH  analysis  includes  cloud  phase  and  cloud  top  height,  which 
can  be  used  to  help  determine  whether  a  conflicting  surface  observation  should  override  a 
satellite-derived  cloud  analysis. 


10.  TACNEPH  Computer  Program 

The  TACNEPH  program  had  requirements  in  two  areas: 

1)  development  of  algorithms  capable  of  generating  gridded  fields  of  cloud  amount 
and  altitude  from  data  sources  available  in  a  tactical  terminal  and 

2)  implementation  of  those  algorithms  as  research  grade  FORTRAN  code  with 
sufficient  documentation  to  support  transition  to  operational  programs  on  the 
Mark  IV-B. 

In  previous  sections,  this  report  has  primarily  addressed  the  first  requirement  area. 
However,  a  considerable  part  of  the  TACNEPH  program  was  devoted  to  development  of 
working  software  to  implement  the  new  algorithm  in  a  working  environment  (see  for 
example  the  discussion  on  algorithm  testing  and  validation  in  Section  5.3).  In  this  section 
we  provide  a  brief  description  of  that  process  with  the  assumption  that  "lessons  learned" 
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will  be  of  value  during  operational  implementation  on  a  tactical  terminal.  Detail  on  the 
individual  software  modules  is  provided  separately  in  the  relevant  software  design  reports. 

TACNEPH  processing  requirements  were  designed  to  be  consistent  with  typical 
low-end  mini-computers,  although,  depending  on  the  implementation,  the  algorithm  can  be 
memory  intensive.  The  research  grade  code  was  developed  to  run  in  batch  mode  on  a 
Micro-VAX  3900  with  20  MB  of  memory  and  interactively  on  a  VAX-station  3600  with  16 
MB  of  memory..  Both  systems  run  VMS  version  5.5  and  the  software  makes  extensive  use 
of  the  virtual  memory  capabilities  of  the  operating  system.  Currently  the  program  requires 
approximately  25  MB  of  virtual  memory  and  can  process  a  typical  IK  by  IK  image  in 
about  8  minutes  of  CPU  time  running  batch  mode.  T5q)ically  data  are  processed  in  32  x  32 
pixel  arrays  to  approximate  a  16th  mesh  grid  size,  this  translates  into  memory  requirements 
for  storage  of  32  lines  of  sensor  and  supporting  data  plus  diagnostic  information  and  the 
internal  results  buffer.  This  can  and  should  be  optimized  to  acconunodate  specific 
constraints  of  the  target  tactical  terminal  processing  environment.  These  numbers  are 
provided  as  guidelines  only  and  should  not  be  interpreted  as  hard  requirements  for  the 
operational  program  since  the  research  grade  code  is  not  optimized  for  efficiency  in  either 
memory  or  CPU  usage. 

10.1  Research  Grade  Code 

During  the  TACNEPH  development  program,  and  at  program  reviews,  there  was 
considerable  confusion  about  the  definition  of  Research  Grade  Code  (RGC).  The  stated 
requirement  for  tasks  that  required  software  development  was  that  all  software  shall  be 
implemented  as  Research  Grade  Code.  In  this  application,  RGC  is  defined  as  software 
written  to  run  on  the  AIMS  computer  system  for  Ae  sole  purpose  of  demonstrating, 
testing,  and  validating  satellite  data  processing  algorithms  developed  under  TACbffiPH. 

No  attempts  were  made  to  optimize  the  software  to  meet  operational  processing 
requirements.  Several  internal  data  buffers  are  maintained  solely  to  provide  a  record  of 
intermediate  analysis  products  for  use  as  a  diagnostic  of  algorithm  performance. 

Necessarily,  the  TACNEPH  computer  program  was  closely  tied  to  the  TACNEPH 
Database  (Section  3.1)  to  provide  all  data  management  functions.  Database  management 
was  the  most  important  implementation  consideration  since  TACNEPH  operates  on  large 
satellite  databases  that  are  frequently  updated  (Section  2).  In  fact,  much  of  the  required 
code  is  dedicated  to  the  data  management  function.  It  is  implicitly  assumed  that  whatever 
tactical  platform  the  TACNEPH  algorithms  are  eventually  implemented  on  will  have  an 
advanced  capability  to  manage  satellite  data.  Consequently  the  TACNEPH  code  will  be 
tailored  to  accommodate  that  data  management  strategy. 

TACNEPH  RGC  is  useful  as  documentation  in  addition  to  the  algorithm 
descriptions  contained  in  this  report.  It  provides  examples  of  one  way  to  implement  the 
cloud  algorithms.  However,  it  is  emphasized  that  RGC  software  should  not  be  considered 
as  a  model  for  operational  implementation.  It  was  developed  to  meet  the  unique 
requirements  of  the  TACNEPH  research  program  and,  as  such,  contains  many  features  that 
would  likely  be  superfluous  in  an  operational  implementation. 

10.2  Implementation  on  the  AIMS  System 

This  section  is  intended  to  provide  a  general  synopsis  of  the  implementation  of  the 
TACNEPH  computer  algorithm  on  AIMS.  A  detailed  description  of  the  software  is 
provided  in  the  separate  software  design  reports  that  accompany  the  OLS,  AVHRR,  and 
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dear-scene  background  characterization  computer  programs.  The  data  analysis  process 
consists  of  five  separate  components: 

1)  collection  and  calibration  of  satellite  data  via  the  two  satellite  ground  stations, 

2)  ingest  of  the  data  into  TDB, 

3)  apphcation  of  the  cloud  detection  algorithms, 

4)  updating  of  the  infrared  and  visible  dynamic  supporting  databases,  and 

5)  manual  analysis  of  the  results. 

All  steps  except  the  manual  analysis  are  automated  and  require  no  user  input.  The 
algorithm  is  triggered  by  receipt  of  a  satellite  overpass  by  one  of  the  PL  ground  station 
computers. 

The  first  step  in  the  TACNEPH  cloud  detection  scheme  on  AIMS  is  collection  of 
the  satellite  data.  Recall  that  two  ground  stations  are  available  to  collect  DMSP/RTD  and 
NOAA/HRPT  data  in  real-time.  Every  pass  within  the  ground  station  line-of-site  is 
collected  for  NOAA-1 1,  NOAA-12  and  F-1 1  on  a  continuing  basis.  Included  with  the 
ground  station  system  is  software  to  handle  automatic  scheduling  of  data  capture  and 
ingest,  unpacking  of  the  raw  data  stream,  and  calibration  of  the  sensor  data.  When  a  pass 
is  collected,  a  command  script  is  automatically  run  which  initiates  the  TACNEPH 
algorithm.  The  data  stream  is  written  both  to  hard  disk  and  to  permanent  storage  on  tape. 
Next,  the  relevant  sensor  data  (OLS  or  AVHRR)  are  unpacked  from  the  data  stream  and 
calibrated.  At  this  point,  the  data  are  in  a  ground  station-specific  format.  The  imagery  can 
be  displayed  for  the  purposes  of  a  manual  check  of  the  data  quality.  The  data  are  then 
reformatted  for  ingest  into  TDB.  OLS  fine  mode  data  are  sampled  to  match  the  resolution 
of  the  smooth  mode  channel.  Each  sensor  channel  image  is  stored  as  a  separate  file 
(Section  3.1).  Recall  that  the  TACNEPH  cloud  analysis  is  performed  on  scan-formatted 
data  so  that  no  sampling  of  the  original  data  is  required.  In  addition  to  the  sensor  channel 
imagery,  files  of  sampled  latitude,  longitude  and  solar/satellite  angles  are  also  created. 
Using  additional  software  written  expressly  to  manage  the  transfer  of  data  from  the  ground 
stations  to  AIMS,  the  imagery,  supporting  ephemeris  and  satellite  pass  information  are 
copied  into  a  temporary  file  space  on  AIMS.  When  the  transfer  is  complete,  a  TDB  ingest 
script  is  initiated. 

The  ingest  of  data  into  TDB  involves  creating  a  data  dictionary  entry  for  each  file 
and  moving  the  files  from  temporary  to  permanent  space  on  the  AIMS  hard  disk.  TDB  is 
detailed  in  Section  3. 

Once  the  data  are  registered  in  TDB,  the  automated  script  executes  the  satellite 
appropriate  cloud  algorithm.  As  discussed  in  Section  5,  the  cloud  algorithms  handle  the 
graceful  degradation  issue  internally.  A  series  of  paths  through  the  code  are  traversed 
according  to  solar  illumination  constraints,  data  availability  and  quality.  Note  that 
automated  quality  checks  are  limited  to  detection  of  missing  data  and  do  not  include  checks 
for  anomalous  data.  Cloud  detection  results  are  stored  in  permanent  files  and  registered 
with  TDB. 

Using  the  newly  created  data  files  which  contain  cloud  analyses  for  this  satellite 
pass,  dear-scene  visible  and  infrared  pbcel  values  are  identified  and  the  visible  and  infrared 
dynamic  databases  described  in  Section  4. 1  and  4.2  are  updated.  At  this  point,  the 
automated  process  ends. 
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The  final  step  is  manual  analysis  of  the  cloud  detection  results  using  the  interactive 

techniques  described  in  Section  5.3.1. 
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